Jinzhe Zeng's Blog

苟利国家生死以,岂因祸福避趋之

0%

Install TensorFlow’s C++ interface

The tensorflow’s C++ interface will be compiled from the source code. Firstly one installs bazel. The bazel version 3.1.0 should be used. A full instruction of bazel installation can be found here.

1
2
3
4
5
cd /some/workspace
wget https://github.com/bazelbuild/bazel/releases/download/3.1.0/bazel-3.1.0-installer-linux-x86_64.sh
chmod +x bazel-3.1.0-installer-linux-x86_64.sh
./bazel-3.1.0-installer-linux-x86_64.sh --prefix /some/workspace/bazel
export PATH=/some/workspace/bazel/bin:$PATH

Firstly get the source code of the tensorflow

1
2
3
git clone https://github.com/tensorflow/tensorflow tensorflow -b v2.3.0 --depth=1
cd tensorflow
./configure

You will answer a list of questions that help configure the building of tensorflow. You may want to answer the question like the following. If you do not want to add CUDA support, please answer no.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Please specify the location of python. [Default is xxx]:

Found possible Python library paths:
xxx
Please input the desired Python library path to use. Default is [xxx]

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Do you wish to build TensorFlow with TensorRT support? [y/N]:
No TensorRT support will be enabled for TensorFlow.

Found CUDA 10.2 in:
/usr/local/cuda/lib64
/usr/local/cuda/include
Found cuDNN 7 in:
/usr/local/cuda/lib64
/usr/local/cuda/include

Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 7.5,7.5]:

Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
--config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished

The library path for Python should be set accordingly.

Now build the shared library of tensorflow:

1
bazel build -c opt --verbose_failures //tensorflow:libtensorflow_cc.so

You may want to add options --copt=-msse4.2, --copt=-mavx, --copt=-mavx2 and --copt=-mfma to enable SSE4.2, AVX, AVX2 and FMA SIMD accelerations, respectively. It is noted that these options should be chosen according to the CPU architecture. If the RAM becomes an issue of your machine, you may limit the RAM usage by using --local_resources 2048,.5,1.0.

Now I assume you want to install tensorflow in directory $tensorflow_root. Create the directory if it does not exists

1
mkdir -p $tensorflow_root

Now, copy the libraries to the tensorflow’s installation directory:

1
2
3
4
mkdir -p $tensorflow_root/lib
cp -d bazel-bin/tensorflow/libtensorflow_cc.so* $tensorflow_root/lib/
cp -d bazel-bin/tensorflow/libtensorflow_framework.so* $tensorflow_root/lib/
cp -d $tensorflow_root/lib/libtensorflow_framework.so.2 $tensorflow_root/lib/libtensorflow_framework.so

Then copy the headers

1
2
3
4
5
6
7
8
9
mkdir -p $tensorflow_root/include/tensorflow
rsync -avzh --exclude '_virtual_includes/' --include '*/' --include '*.h' --include '*.inc' --exclude '*' bazel-bin/ $tensorflow_root/include/
rsync -avzh --include '*/' --include '*.h' --include '*.inc' --exclude '*' tensorflow/cc $tensorflow_root/include/tensorflow/
rsync -avzh --include '*/' --include '*.h' --include '*.inc' --exclude '*' tensorflow/core $tensorflow_root/include/tensorflow/
rsync -avzh --include '*/' --include '*' --exclude '*.cc' third_party/ $tensorflow_root/include/third_party/
rsync -avzh --include '*/' --include '*' --exclude '*.txt' bazel-tensorflow/external/eigen_archive/Eigen/ $tensorflow_root/include/Eigen/
rsync -avzh --include '*/' --include '*' --exclude '*.txt' bazel-tensorflow/external/eigen_archive/unsupported/ $tensorflow_root/include/unsupported/
rsync -avzh --include '*/' --include '*.h' --include '*.inc' --exclude '*' bazel-tensorflow/external/com_google_protobuf/src/google/ $tensorflow_root/include/google/
rsync -avzh --include '*/' --include '*.h' --include '*.inc' --exclude '*' bazel-tensorflow/external/com_google_absl/absl/ $tensorflow_root/include/absl/

Troubleshooting

1
git: unknown command -C ...

This may be your git version issue, because low version of git does not support this command. Upgrading your git maybe helpful.

每个知乎答主都有退乎的梦想,但退乎前如果删光回答,则十分可惜。因此,我用Python写了60行的脚本,可以在退乎前备份自己的所有回答和文章,以免事后后悔。

GitHub - njzjz/zhihubackup

安装

1
pip install git+https://github.com/njzjz/zhihubackup

使用

假如你是@贱贱,你的id是splitter,那么可以编写Python脚本:

1
2
from zhihubackup import backup_zhihu
backup_zhihu("splitter")

静等一段时间。运行结束后,可以看到产生了名为splitter的文件夹:

1
2
3
4
5
6
- splitter
|- answer (842 files)
|- article (101 files)
|- pin (3214 files)
|- question (57 files)

备份已经成功,现在可以删光回答和文章了。

Cover

2019年5月,林峰邀请我暑假到北大实习,我欣然答应。拿到学士学位证书后,我在家里休息了一周,顺便处理了升学的各项事宜。这些全部搞定后,6月下旬,我启程前往北京。

Read more »

宣传稿在微信的传播经历

统计了论文宣传稿在微信上的传播路径(图),产生几个有意思的结论。一是学校媒体和传统媒体的影响力仍然是最大的,应当作为宣传时优先考虑的对象;二是自媒体喜欢抄来抄去,扩大了宣传稿的阅读量;三是一些媒体乱改标题,总喜欢搞个大新闻。

转载自华东师范大学新闻网

华东师范大学化学与分子工程学院朱通副研究员指导学生结合人工智能算法、量子化学理论以及分子动力学方法,实现了燃料燃烧的高精度计算机模拟,在原子尺度和亚飞秒时间分辨率下获得了甲烷燃烧的化学反应网络。该成果以“Complex reaction processes in combustion unraveled by neural network-based molecular dynamics simulation”为题近日发表在Nature Communications [Nat. Commun. 2020, 11, 5713.]。华东师范大学为论文第一单位,2019届本科毕业生曾晋哲为论文的第一作者,朱通副研究员和张增辉教授为论文的共同通讯作者。

Read more »

今日,华东师范大学化学与分子工程学院(下文简称“学院”)在其官方网站发布《致用人单位的一封信》。文中指出,学院2020届毕业生就业工作表现优异,就业率和就业质量成绩显著;而2021届毕业生就业形势更为严峻,学院邀请各用人单位选聘毕业生。

笔者此前发现,学院2018届就业率仅85.66%,为2015届以来最低;但2019届就业率达98.78%,超过全校平均水平,其中本科生就业率达100%。今年9月,学院获学生就业工作先进集体称号。

Read more »

很多人对Anaconda有这样的误解,认为conda只是用来管理Python packages的。实际上,conda更神奇的功能,是可以打包编译后的程序,并且不需要root权限即可安装。一般来说,分发编译后的程序总有各种各样的静态库的依赖问题,所以Anaconda把这些依赖库也做成了conda packages,这样就能确保任何机器都能安装了。libtensorflow_cc是TensorFlow的C++库,之前介绍过如何编译,但编译成conda packages,仍需要费一番功夫。本文介绍2.3版本的构建方法:

我们先来看deepmd-kit-recipes/libtensorflow_cc-feedstockrecipe/build.sh文件。conda-build构建时,会默认把build.sh作为编译脚本。

1
2
3
4
5
6
if [ ${cuda_compiler_version} == "None" ]; then
bash $RECIPE_DIR/build_cpu.sh
fi
if [ ${cuda_compiler_version} != "None" ]; then
bash $RECIPE_DIR/build_gpu.sh
fi
Read more »