当前位置： 首页 > news >正文

安装和使用 nn-Meter

news 2025/8/12 7:35:18

安装和使用 nn-Meter

nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
nn-Meter：精准预测深度学习模型在边缘设备上的推理延迟

Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu

1. nn-Meter

https://github.com/microsoft/nn-Meter

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices. The key idea is dividing a whole model inference into kernels, i.e., the execution units of fused operators on a device, and conduct kernel-level prediction. We currently evaluate four popular platforms on a large dataset of 26k models. It achieves 99.0% (mobile CPU), 99.1% (mobile Adreno 640 GPU), 99.0% (mobile Adreno 630 GPU), and 83.4% (Intel VPU) prediction accuracy.
为了高效、准确地预测深度神经网络模型在不同边缘设备上的推理延迟，作者提出并开发了一个基于内核的模型推理延迟预测系统 nn-Meter，引入了内核检测，可找出算子融合行为。通过对最有价值的数据进行采样，nn-Meter 有效地建立了内核的延迟预测器。

The current supported hardware and inference frameworks:

DeviceFrameworkProcessor±10% AccuracyHardware name
Pixel4TFLite v2.1CortexA76 CPU99.0%cortexA76cpu_tflite21
Mi9TFLite v2.1Adreno 640 GPU99.1%adreno640gpu_tflite21
Pixel3XLTFLite v2.1Adreno 630 GPU99.0%adreno630gpu_tflite21
Intel Movidius NCS2OpenVINO2019R2Myriad VPU83.4%myriadvpu_openvino2019r2

Tags
https://github.com/microsoft/nn-Meter/tags

Releases
https://github.com/microsoft/nn-Meter/releases

https://github.com/microsoft/nn-Meter/releases/tag/v1.0-data

adreno630gpu_tflite21.zip
adreno640gpu_tflite21.zip
cortexA76cpu_tflite21.zip
datasets.zip
ir_graphs.zip
myriadvpu_openvino2019r2.zip
onnx_models.zip
pb_models.zip
Source code (zip)
Source code (tar.gz)

https://github.com/microsoft/nn-Meter/releases/tag/v2.0-data

tflite_benchmark_tools_v2.1.zip
tflite_benchmark_tools_v2.7.zip
Source code (zip)
Source code (tar.gz)

nn-Meter Builder
https://github.com/microsoft/nn-Meter/blob/main/docs/builder/overview.md

Neural Network Intelligence，NNI
https://github.com/microsoft/nni
https://www.microsoft.com/en-us/research/project/neural-network-intelligence/
https://nni.readthedocs.io/zh/stable/

NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments.

26k latency benchmark dataset
https://github.com/microsoft/nn-Meter/releases/download/v1.0-data/datasets.zip

2. Installation

Currently nn-Meter has been tested on Linux and Windows system. Windows 10, Ubuntu 16.04 and 20.04 with python 3.6.10 are tested and supported. Please first install python3 before nn-Meter installation. Then nn-Meter could be installed by running:

pip install nn-meter
pip install nn-meter --default-timeout=1000 -i https://pypi.tuna.tsinghua.edu.cn/simple

nn-meter==2.0 has been released now.

(base) yongqiang@yongqiang:~/yongqiang_work$ pip install nn-meter --default-timeout=1000 -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: nn-meter in /home/yongqiang/miniconda3/lib/python3.11/site-packages (2.0)
Requirement already satisfied: numpy in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (1.25.2)
Requirement already satisfied: pandas in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (1.5.3)
Requirement already satisfied: tqdm in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (4.65.0)
Requirement already satisfied: networkx in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (3.2.1)
Requirement already satisfied: requests in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (2.31.0)
Requirement already satisfied: protobuf in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (4.25.0)
Requirement already satisfied: PyYAML in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (6.0.1)
Requirement already satisfied: scikit-learn in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (1.3.2)
Requirement already satisfied: packaging in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (23.0)
Requirement already satisfied: jsonlines in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from nn-meter) (4.0.0)
Requirement already satisfied: attrs>=19.2.0 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from jsonlines->nn-meter) (23.1.0)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from pandas->nn-meter) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from pandas->nn-meter) (2022.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from requests->nn-meter) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from requests->nn-meter) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from requests->nn-meter) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from requests->nn-meter) (2023.7.22)
Requirement already satisfied: scipy>=1.5.0 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from scikit-learn->nn-meter) (1.11.3)
Requirement already satisfied: joblib>=1.1.1 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from scikit-learn->nn-meter) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from scikit-learn->nn-meter) (3.2.0)
Requirement already satisfied: six>=1.5 in /home/yongqiang/miniconda3/lib/python3.11/site-packages (from python-dateutil>=2.8.1->pandas->nn-meter) (1.16.0)
(base) yongqiang@yongqiang:~/yongqiang_work$

If you want to try latest code, please install nn-Meter from source code. First git clone nn-Meter package to local:

git clone git@github.com:microsoft/nn-Meter.git
cd nn-Meter

Then simply run the following pip install in an environment that has python >= 3.6. The command will complete the automatic installation of all necessary dependencies and nn-Meter.

pip install .

nn-Meter is a latency predictor of models with type of Tensorflow, PyTorch, Onnx, nn-meter IR graph and NNI IR graph (https://github.com/microsoft/nni). To use nn-Meter for specific model type, you also need to install corresponding required packages. The well tested versions are listed below:

Testing Model TypeRequirements
Tensorflowtensorflow==2.6.0
Torchtorch==1.9.0, torchvision==0.10.0, (alternative) [onnx>=1.9.0, onnx-simplifier==0.3.6] or [nni>=2.4]
Onnxonnx==1.9.0
nn-Meter IR graph—
NNI IR graphnni>=2.4

Please also check the versions of numpy and scikit_learn. The different versions may change the prediction accuracy of kernel predictors.

The stable version of wheel binary package will be released soon.

3. Usage

To apply for hardware latency prediction, nn-Meter provides two types of interfaces：

  • command line nn-meter after nn-meter installation.
  • Python binding provided by the module nn_meter

Here is a summary of supported inputs of the two methods.

Testing Model TypeCommand SupportPython Binding
TensorflowCheckpoint file dumped by tf.saved_model() and end with .pbCheckpoint file dumped by tf.saved_model and end with .pb
TorchModels in torchvision.modelsObject of torch.nn.Module
OnnxCheckpoint file dumped by torch.onnx.export() or onnx.save() and end with .onnxCheckpoint file dumped by onnx.save() or model loaded by onnx.load()
nn-Meter IR graphJson file in the format of nn-Meter IR Graphdict object following the format of nn-Meter IR Graph
NNI IR graph-NNI IR graph object

In both methods, users could appoint predictor name and version to target a specific hardware platform (device). Currently, nn-Meter supports prediction on the following four configs:

Predictor (device_inference_framework)Processor CategoryVersion
cortexA76cpu_tflite21CPU1.0
adreno640gpu_tflite21GPU1.0
adreno630gpu_tflite21GPU1.0
myriadvpu_openvino2019r2VPU1.0

Users can get all predefined predictors and versions by running

# all predefined predictors
nn-meter --list-predictors

(base) yongqiang@yongqiang:~/yongqiang_work$ nn-meter --list-predictors
(nn-Meter) Supported latency predictors:
(nn-Meter) [Predictor] cortexA76cpu_tflite21: version=1.0
(nn-Meter) [Predictor] adreno640gpu_tflite21: version=1.0
(nn-Meter) [Predictor] adreno630gpu_tflite21: version=1.0
(nn-Meter) [Predictor] myriadvpu_openvino2019r2: version=1.0
(base) yongqiang@yongqiang:~/yongqiang_work$

(base) yongqiang@yongqiang:~/yongqiang_work$ nn-meter -h
usage: nn-meter [-h] [-v] [--list-predictors] [--list-backends] [--list-kernels] [--list-operators] [--list-testcases]{predict,lat_pred,get_ir,create,connect,register,unregister} ...please run "nn-meter {positional argument} --help" to see nn-meter guidancepositional arguments:{predict,lat_pred,get_ir,create,connect,register,unregister}predict (lat_pred)  apply latency predictor for testing modelget_ir              specify a model type to convert to nn-meter ir graphcreate              create a workspace folder for nn-Meter builderconnect             connect to backendregister            register customized module to nn-Meter, supporting type: predictor, backend, operator, testcase, operatorunregister          unregister customized module from nn-Meter, supporting type: predictor, backend, operator, testcase, operatoroptions:-h, --help            show this help message and exit-v, --verbose         increase output verbosity--list-predictors     list all supported predictors--list-backends       list all supported backends--list-kernels        list all supported kernels when building kernel predictors--list-operators      list all supported operators when building fusion rule test cases--list-testcases      list all supported special test cases when building fusion rule test cases
(base) yongqiang@yongqiang:~/yongqiang_work$

3.1. Predict latency of saved CNN model

After installation, a command named nn-meter is enabled. To predict the latency for a CNN model with a predefined predictor in command line, users can run the following commands (sample models can be downloaded here)

# for Tensorflow (*.pb) file
nn-meter predict --predictor <hardware> [--predictor-version <version>] --tensorflow <pb-file_or_folder> 
# Example Usage
nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --tensorflow mobilenetv3small_0.pb # for ONNX (*.onnx) file
nn-meter predict --predictor <hardware> [--predictor-version <version>] --onnx <onnx-file_or_folder>
# Example Usage
nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --onnx mobilenetv3small_0.onnx # for torch model from torchvision model zoo
nn-meter predict --predictor <hardware> [--predictor-version <version>] --torchvision <model-name> <model-name>... 
# Example Usage
nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --torchvision mobilenet_v2# for nn-Meter IR (*.json) file
nn-meter predict --predictor <hardware> [--predictor-version <version>] --nn-meter-ir <json-file_or_folder> 
# Example Usage
nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --nn-meter-ir mobilenetv3small_0.json

--predictor-version <version> arguments is optional. When the predictor version is not specified by users, nn-meter will use the latest version of the predictor.

(base) yongqiang@yongqiang:~/yongqiang_work/nn-Meter/material/testmodels$ pwd
/home/yongqiang/yongqiang_work/nn-Meter/material/testmodels
(base) yongqiang@yongqiang:~/yongqiang_work/nn-Meter/material/testmodels$ ls -l
total 18080
-rw-r--r-- 1 yongqiang yongqiang   239602 Nov 12 10:13 mobilenetv3small_0.json
-rw-r--r-- 1 yongqiang yongqiang 10169165 Nov 12 10:13 mobilenetv3small_0.onnx
-rw-r--r-- 1 yongqiang yongqiang  8098911 Nov 12 10:13 mobilenetv3small_0.pb
(base) yongqiang@yongqiang:~/yongqiang_work/nn-Meter/material/testmodels$

nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --tensorflow /home/yongqiang/yongqiang_work/nn-Meter/material/testmodels/mobilenetv3small_0.pb

nn-Meter can support batch mode prediction. To predict latency for multiple models in the same model type once, user should collect all models in one folder and state the folder after --[model-type] liked argument.

It should also be noted that for PyTorch model, nn-meter can only support existing models in torchvision model zoo. The string followed by --torchvision should be exactly one or more string indicating name(s) of some existing torchvision models. To apply latency prediction for torchvision model in command line, onnx and onnx-simplifier packages are required.

3.2. Convert to nn-Meter IR Graph

Furthermore, users may be interested to convert tensorflow pb-file or onnx file to nn-Meter IR graph. Users could convert nn-Meter IR graph and save to .json file be running

# for Tensorflow (*.pb) file
nn-meter get_ir --tensorflow <pb-file> [--output <output-name>]# for ONNX (*.onnx) file
nn-meter get_ir --onnx <onnx-file> [--output <output-name>]

Output name is default to be /path/to/input/file/<input_file_name>_<model-type>_ir.json if not specified by users.

3.3. Use nn-Meter in your Python code

After installation, users can import nn-Meter in python code

from nn_meter import load_latency_predictorpredictor = load_latency_predictor(hardware_name, hardware_predictor_version) # case insensitive in backend# build your model (e.g., model instance of torch.nn.Module)
model = ... lat = predictor.predict(model, model_type) # the resulting latency is in unit of ms

By calling load_latency_predictor, user selects the target hardware and loads the corresponding predictor. nn-Meter will try to find the right predictor file in ~/.nn_meter/data. If the predictor file doesn’t exist, it will download from the Github release.

In predictor.predict(), the allowed items of the parameter model_type include ["pb", "torch", "onnx", "nnmeter-ir", "nni-ir"], representing model types of tensorflow, torch, onnx, nn-meter IR graph and NNI IR graph, respectively.

For Torch models, the shape of feature maps is unknown merely based on the given network structure, which is, however, significant parameters in latency prediction. Therefore, torch model requires a shape of input tensor for inference as a input of predictor.predict(). Based on the given input shape, a random tensor according to the shape will be generated and used. Another thing for Torch model prediction is that users can install the onnx and onnx-simplifier packages for latency prediction (referred to as Onnx-based latency prediction for torch model), or alternatively install the nni package (referred to as NNI-based latency prediction for torch model). Note that the nni option does not support command line calls. In addition, if users use nni for latency prediction, the PyTorch modules should be defined by the nn interface from NNI import nni.retiarii.nn.pytorch as nn (view NNI Documentation (https://nni.readthedocs.io/en/stable/) for more information), and the parameter apply_nni should be set as True in the function predictor.predict(). Here is an example of NNI-based latency prediction for Torch model:

import nni.retiarii.nn.pytorch as nn
from nn_meter import load_latency_predictorpredictor = load_latency_predictor(...)# build your model using nni.retiarii.nn.pytorch as nn
model = nn.Module ...input_shape = (1, 3, 224, 224)
lat = predictor.predict(model, model_type='torch', input_shape=input_shape, apply_nni=True)

The Onnx-based latency prediction for torch model is stable but slower, while the NNI-based latency prediction for torch model is unstable as it could fail in some case but much faster compared to the Onnx-based model. The Onnx-based model is set as the default one for Torch model latency prediction in nn-Meter. Users could choose which one they preferred to use according to their needs.

Users could view the information all built-in predictors by list_latency_predictors or view the config file in nn_meter/configs/predictors.yaml.

Users could get a nn-Meter IR graph by applying model_file_to_graph and model_to_graph by calling the model name or model object and specify the model type. The supporting model types of model_file_to_graph include “onnx”, “pb”, “torch”, “nnmeter-ir” and “nni-ir”, while the supporting model types of model_to_graph include “onnx”, “torch” and “nni-ir”.

3.4. nn-Meter Builder

nn-Meter builder is an open source tool for users to build latency predictor on their own devices. There are three main parts in nn-Meter builder:

backend: the module of connecting backends;

backend_meta: the meta tools related to backend. Here we provide the fusion rule tester to detect fusion rules for users’ backend;

kernel_predictor_builder: the tool to build different kernel latency predictors.

Users could get access to nn-Meter builder by calling nn_meter.builder. For more details to use nn-Meter builder, please check the document of nn-Meter builder.

3.5. Hardware-aware NAS by nn-Meter and NNI

To empower affordable DNN on the edge and mobile devices, hardware-aware NAS searches both high accuracy and low latency models. In particular, the search algorithm only considers the models within the target latency constraints during the search process.

empower [ɪmˈpaʊə(r)]：v. 授权，给 (某人) ... 的权力，增加 (某人的) 自主权，使控制局势
affordable [əˈfɔː(r)dəb(ə)l]：adj. 负担得起的

Currently we provide two examples of hardware-aware NAS, including end-to-end multi-trial NAS (https://nni.readthedocs.io/en/stable/) which is a random search algorithm (https://arxiv.org/abs/1902.07638) on SPOS NAS (Single Path One-Shot Neural Architecture Search with Uniform Sampling) search space, and the popular ProxylessNAS (https://nni.readthedocs.io/en/stable/), which is a one-shot NAS algorithm with hardware-efficient loss function. More examples of other widely-used hardware-aware NAS and model compression algorithms are coming soon.

3.5.1. Multi-trial SPOS Demo

To run multi-trail SPOS demo, NNI should be installed through source code by following NNI Documentation (https://nni.readthedocs.io/en/stable/)

python setup.py develop

Then run multi-trail SPOS demo:

python ${NNI_ROOT}/examples/nas/oneshot/spos/multi_trial.py
  • How the demo works

Refer to NNI Documentation (https://nni.readthedocs.io/en/stable/) for how to perform NAS by NNI.

To support hardware-aware NAS, you first need a Strategy that supports filtering the models by latency. We provide such a filter named LatencyFilter in NNI and initialize a Random strategy with the filter:

simple_strategy = strategy.Random(model_filter=LatencyFilter(threshold=100, predictor=base_predictor))

LatencyFilter will predict the models’ latency by using nn-Meter and filter out the models whose latency with the given predictor are larger than the threshold (i.e., 100 in this example).
You can also build your own strategies and filters to support more flexible NAS such as sorting the models according to latency.

Then, pass this strategy to RetiariiExperiment:

exp = RetiariiExperiment(base_model, trainer, strategy=simple_strategy)exp_config = RetiariiExeConfig('local')
...
exp_config.dummy_input = [1, 3, 32, 32]exp.run(exp_config, port)

In exp_config, dummy_input is required for tracing shape info.

3.5.2. ProxylessNAS Demo

To run the one-shot ProxylessNAS demo, users can run the NNI ProxylessNAS training demo:

python ${NNI_ROOT}/examples/nas/oneshot/proxylessnas/main.py --applied_hardware <hardware> --reference_latency <reference latency (ms)>
  • How the demo works

Refer to NNI Documentation (https://nni.readthedocs.io/en/stable/) for how to perform NAS by NNI.

ProxylessNAS currently builds a lookup table, that stores the measured latency of each candidate building block in the search space. The latency sum of all building blocks in a candidate model will be treated as the model inference latency. With leveraging nn-Meter in NNI, users can apply ProxylessNAS to search efficient DNN models on more types of edge devices. In NNI implementation, a HardwareLatencyEstimator predicts expected latency for the mixed operation based on the path weight of ProxylessLayerChoice. To call nn-Meter in NNI ProxylessNAS, users can add the arguments of “--applied_hardware <hardware> --reference_latency <reference latency (ms)>” in the example (https://github.com/microsoft/nni/blob/master/examples/nas/legacy/oneshot/proxylessnas/main.py).

3.6. Benchmark Dataset

To evaluate the effectiveness of a prediction model on an arbitrary DNN model, we need a representative dataset that covers a large prediction scope. nn-Meter collects and generates 26k CNN models. (Please refer the paper for the dataset generation method.)

We release the dataset, and provide an interface of nn_meter.dataset for users to get access to the dataset. Users can also download the data from the Download Link (https://github.com/microsoft/nn-Meter/releases/download/v1.0-data/datasets.zip) on their own.

4. Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct (https://opensource.microsoft.com/codeofconduct/).
For more information see the Code of Conduct FAQ (https://opensource.microsoft.com/codeofconduct/faq/) or
contact opencode@microsoft.com with any additional questions or comments.

5. License

The entire codebase is under MIT license (https://github.com/microsoft/nn-Meter/blob/main/LICENSE)

The dataset is under Open Use of Data Agreement (https://github.com/Community-Data-License-Agreements/Releases/blob/main/O-UDA-1.0.md)

6. Citation

If you find that nn-Meter helps your research, please consider citing it:

@inproceedings{nnmeter,author = {Zhang, Li Lyna and Han, Shihao and Wei, Jianyu and Zheng, Ningxin and Cao, Ting and Yang, Yuqing and Liu, Yunxin},title = {nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices},year = {2021},publisher = {ACM},address = {New York, NY, USA},url = {https://doi.org/10.1145/3458864.3467882},doi = {10.1145/3458864.3467882},booktitle = {Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services},pages = {81–93},
}@misc{nnmetercode,author = {Microsoft Research nn-Meter Team},title = {nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices},year = {2021},url = {https://github.com/microsoft/nn-Meter},
}

References

[1] Yongqiang Cheng, https://yongqiang.blog.csdn.net/
[2] microsoft / nn-Meter, https://github.com/microsoft/nn-Meter
[3] nn-meter 2.0, https://pypi.org/project/nn-meter/
[4] nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices, https://air.tsinghua.edu.cn/pdf/nn-Meter-Towards-Accurate-Latency-Prediction-of-Deep-Learning-Model-Inference-on-Diverse-Edge-Devices.pdf

http://www.lryc.cn/news/227791.html

相关文章：

  • PHP原生类总结利用
  • C/C++满足条件的数累加 2021年9月电子学会青少年软件编程（C/C++）等级考试一级真题答案解析
  • zookeeper:服务器有几种状态？
  • 大数据-之LibrA数据库系统告警处理（ALM-12040 系统熵值不足）
  • HTML页面模拟了一个类似Excel的表格在线diy修改表格内容
  • Unity如何保存场景，如何导出工程文件/如何查看保存位置？【各版本通用】
  • 2023年第十六届山东省职业院校技能大赛中职组“网络安全”赛项规程
  • html菜单的基本制作
  • Spark Job优化
  • CSS花边001：无衬线字体和有衬线字体
  • nodejs+vue＋python＋PHP＋微信小程序-安卓- 基于小程序的高校后勤管理系统-计算机毕业设计
  • Leetcode153. Find Minimum in Rotated Sorted Array
  • 为什么要用“交叉熵”做损失函数
  • 【Android】Android apk 逆向编译
  • 04-详解SpringBoot自动装配的原理,依赖属性配置的实现,源码分析
  • [100天算法】-不同路径 III（day 73）
  • 【c++随笔12】继承
  • Excel中使用数据验证、OFFSET实现自动更新式下拉选项
  • Android修行手册 - 可变参数中星号什么作用（冷知识）
  • Python与ArcGIS系列（三）视图缩放
  • [ASP]数据库编辑与管理V1.0
  • MyBatis Plus整合Redis实现分布式二级缓存
  • 如何帮助 3D CAD 设计师实现远程办公
  • 如何在 Idea 中修改文件的字符集（如：UTF-8）
  • 【C++】单例模式【两种实现方式】
  • php的api接口token简单实现
  • CCNA课程实验-13-PPPoE
  • cocosCreator 之 Bundle使用
  • 分类网络搭建示例
  • 为 Ubuntu 虚拟机构建 SSH 服务器