Did you install tensorflow_cc to /usr/local after you compile "sudo make install" ?
It should put the include and lib files in there and you should have no problems.
Once you do that change USE_TF = 1 in the egbbdll Makefile. Right now it is defaulting
to the second approach USE_TF = 2 -- which is probably why you are getting those
errors with .pb.h files.
Btw, you don't need to download egbb files to use neural networks.
Daniel
Scorpio 2.8.7 MCTS+NN windows version
Moderators: hgm, Rebel, chrisw
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
Thank you.Daniel Shawul wrote: ↑Mon Sep 24, 2018 8:01 pm Did you install tensorflow_cc to /usr/local after you compile "sudo make install" ?
It should put the include and lib files in there and you should have no problems.
Once you do that change USE_TF = 1 in the egbbdll Makefile. Right now it is defaulting
to the second approach USE_TF = 2 -- which is probably why you are getting those
errors with .pb.h files.
Btw, you don't need to download egbb files to use neural networks.
Daniel
Nice to know no egbb files are needed.
Although I'm pretty sure I did both, "sudo make install" after compiling
and changing USE_TF = 1, I will give it another try tomorrow.
Jörg Oster
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
The second try yields the exact same result and same problems, unfortunately.Joerg Oster wrote: ↑Mon Sep 24, 2018 10:27 pmThank you.Daniel Shawul wrote: ↑Mon Sep 24, 2018 8:01 pm Did you install tensorflow_cc to /usr/local after you compile "sudo make install" ?
It should put the include and lib files in there and you should have no problems.
Once you do that change USE_TF = 1 in the egbbdll Makefile. Right now it is defaulting
to the second approach USE_TF = 2 -- which is probably why you are getting those
errors with .pb.h files.
Btw, you don't need to download egbb files to use neural networks.
Daniel
Nice to know no egbb files are needed.
Although I'm pretty sure I did both, "sudo make install" after compiling
and changing USE_TF = 1, I will give it another try tomorrow.
Interestingly, the small example (inside the folder 'example') builds without any problems and outputs "Session successfully created."
I found two issues on the github repository, though, which seem to be closely related to my problem.
https://github.com/tensorflow/tensorflow/issues/1890 and
https://github.com/FloopCZ/tensorflow_cc/issues/121
Jörg Oster
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: Scorpio 2.8.7 MCTS+NN windows version
Did you build the shared or static library of tensorflow_cc? egbbdll needs the latter and you should haveJoerg Oster wrote: ↑Tue Sep 25, 2018 11:13 amThe second try yields the exact same result and same problems, unfortunately.Joerg Oster wrote: ↑Mon Sep 24, 2018 10:27 pmDaniel Shawul wrote: ↑Mon Sep 24, 2018 8:01 pm Did you install tensorflow_cc to /usr/local after you compile "sudo make install" ?
It should put the include and lib files in there and you should have no problems.
Once you do that change USE_TF = 1 in the egbbdll Makefile. Right now it is defaulting
to the second approach USE_TF = 2 -- which is probably why you are getting those
errors with .pb.h files.
Btw, you don't need to download egbb files to use neural networks.
Daniel
Thank you.
Nice to know no egbb files are needed.
Although I'm pretty sure I did both, "sudo make install" after compiling
and changing USE_TF = 1, I will give it another try tomorrow.
Interestingly, the small example (inside the folder 'example') builds without any problems and outputs "Session successfully created."
I found two issues on the github repository, though, which seem to be closely related to my problem.
https://github.com/tensorflow/tensorflow/issues/1890 and
https://github.com/FloopCZ/tensorflow_cc/issues/121
libtensorflow_cc and libprotobuf.a in /usr/local/lib/tensorflow
For the shared library, you need to do
cmake -DTENSORFLOW_STATIC=OFF -DTENSORFLOW_SHARED=ON ..
What version of bazel are you using? At some point only 0.11 used to work for me but since you
already built it i guess this is no problem for you. The include directories i have in egbbdll Makefile should
be enough to build.
Also you might want to build and install both the shared and static libraries. I am not sure but initially i may
had similar issues as yours when i only had shared library installed...probably an issue with tensorflow_cc
Sorry i couldn't be of more help.
Daniel
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
Your help is highly appreciated, of course.Daniel Shawul wrote: ↑Tue Sep 25, 2018 3:37 pmDid you build the shared or static library of tensorflow_cc? egbbdll needs the latter and you should haveJoerg Oster wrote: ↑Tue Sep 25, 2018 11:13 amThe second try yields the exact same result and same problems, unfortunately.Joerg Oster wrote: ↑Mon Sep 24, 2018 10:27 pmDaniel Shawul wrote: ↑Mon Sep 24, 2018 8:01 pm Did you install tensorflow_cc to /usr/local after you compile "sudo make install" ?
It should put the include and lib files in there and you should have no problems.
Once you do that change USE_TF = 1 in the egbbdll Makefile. Right now it is defaulting
to the second approach USE_TF = 2 -- which is probably why you are getting those
errors with .pb.h files.
Btw, you don't need to download egbb files to use neural networks.
Daniel
Thank you.
Nice to know no egbb files are needed.
Although I'm pretty sure I did both, "sudo make install" after compiling
and changing USE_TF = 1, I will give it another try tomorrow.
Interestingly, the small example (inside the folder 'example') builds without any problems and outputs "Session successfully created."
I found two issues on the github repository, though, which seem to be closely related to my problem.
https://github.com/tensorflow/tensorflow/issues/1890 and
https://github.com/FloopCZ/tensorflow_cc/issues/121
libtensorflow_cc and libprotobuf.a in /usr/local/lib/tensorflow
For the shared library, you need to do
cmake -DTENSORFLOW_STATIC=OFF -DTENSORFLOW_SHARED=ON ..
What version of bazel are you using? At some point only 0.11 used to work for me but since you
already built it i guess this is no problem for you. The include directories i have in egbbdll Makefile should
be enough to build.
Also you might want to build and install both the shared and static libraries. I am not sure but initially i may
had similar issues as yours when i only had shared library installed...probably an issue with tensorflow_cc
Sorry i couldn't be of more help.
Daniel
I just successfully built the static library on my main PC.
At least, it didn't break with any error. Just the usual warnings.
As described on the tensorflow_cc github page, I strictly followed the commands to build the static version:
I have 3 files inside /usr/local/lib/tensorflow_cc: libprotobuf.a libtensorflow-core.a nsync.amkdir build
cd build
cmake ..
make
sudo make install
Unfortunately, I get the exact same error(s) when trying to compile egbbdll like on my other system.
Code: Select all
In file included from /usr/local/include/tensorflow/tensorflow/core/lib/core/errors.h:21:0,
from /usr/local/include/tensorflow/tensorflow/core/platform/env.h:24,
from eval_nn.cpp:4:
/usr/local/include/tensorflow/tensorflow/core/lib/core/status.h:23:10: fatal error: tensorflow/core/lib/core/error_codes.pb.h: Datei oder Verzeichnis nicht gefunden
#include "tensorflow/core/lib/core/error_codes.pb.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Question: If you were able to successfully compile egbbdll on your Linux system, shouldn't this also run on my box?
Maybe this is worth a try?
Jörg Oster
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: Scorpio 2.8.7 MCTS+NN windows version
Yes, I can upload the linux-cpu version.
Egbbdll does not currently build with the static version of tensorflow_cc. It used to but I removed the feature a month ago
because I wanted to have a separate tensorflow so file (though i have the opposite for windows version now).
The only reason I asked you to try and build the static library is because i thought maybe it will put the missing headers in
/usr/local/include/tensorflow. I remember i needed to build both initially for some reason ... So build the shared tensorflow_cc on top of the
static one if you haven't already done so.
I will upload the egbbdll and libtensorflow_cc later for you to try if that doesn't work.
Daniel
Egbbdll does not currently build with the static version of tensorflow_cc. It used to but I removed the feature a month ago
because I wanted to have a separate tensorflow so file (though i have the opposite for windows version now).
The only reason I asked you to try and build the static library is because i thought maybe it will put the missing headers in
/usr/local/include/tensorflow. I remember i needed to build both initially for some reason ... So build the shared tensorflow_cc on top of the
static one if you haven't already done so.
I will upload the egbbdll and libtensorflow_cc later for you to try if that doesn't work.
Daniel
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
OK, building the shared version atm.Daniel Shawul wrote: ↑Tue Sep 25, 2018 6:00 pm Yes, I can upload the linux-cpu version.
Egbbdll does not currently build with the static version of tensorflow_cc. It used to but I removed the feature a month ago
because I wanted to have a separate tensorflow so file (though i have the opposite for windows version now).
The only reason I asked you to try and build the static library is because i thought maybe it will put the missing headers in
/usr/local/include/tensorflow. I remember i needed to build both initially for some reason ... So build the shared tensorflow_cc on top of the
static one if you haven't already done so.
I will upload the egbbdll and libtensorflow_cc later for you to try if that doesn't work.
Daniel
Will report later if that worked.
Jörg Oster
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
Finally success!
After building the shared library, which took quite some time,
compiling egbbdll worked (with USE_TF = 1).
Only thing I needed to change was line 56 of the egbbdll makefile into
Now I hope the fun part can start!
Thanks for your help!
After building the shared library, which took quite some time,
compiling egbbdll worked (with USE_TF = 1).
Only thing I needed to change was line 56 of the egbbdll makefile into
Code: Select all
TF_LIB += $(TF_DIR_LIB)/libtensorflow-core.a
Thanks for your help!
Jörg Oster
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: Scorpio 2.8.7 MCTS+NN windows version
Ok nice ! It seems you are using the static library though. Here is the commit as to when I removed it -- but maybe it is time to bring it back.Joerg Oster wrote: ↑Tue Sep 25, 2018 9:24 pm Finally success!
After building the shared library, which took quite some time,
compiling egbbdll worked (with USE_TF = 1).
Only thing I needed to change was line 56 of the egbbdll makefile intoNow I hope the fun part can start!Code: Select all
TF_LIB += $(TF_DIR_LIB)/libtensorflow-core.a
Thanks for your help!
https://github.com/dshawul/egbbdll/comm ... f6fd6ed5e6
The shared library built with --config=monolithic is libtensorflow_cc.so and this is the one that can be used with GPUs.
Also there is an inherent problem with protobuf and tensorflow shared library built outside of bazel.
I was not able to use multiple GPUs with tensorflow_cc and other prebuilt windows libraries as well -- and I spent
literally weeks chasing for a workaround. I couldn't find a solution so i resorted to building using bazel.
Your egbbdll binary will crash because of that where it tries to set the CPU/GPU number, so you need to comment out the following lines
in eval_nn.cpp
Code: Select all
//std::string dev_name = ((dev_type == GPU) ? "/gpu:" : "/cpu:") + std::to_string(dev_id);
//graph::SetDefaultDevice(dev_name, &graph_def);
Daniel
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Scorpio 2.8.7 MCTS+NN windows version
Unfortunately, the static build of egbbdll doesn't load.Daniel Shawul wrote: ↑Tue Sep 25, 2018 10:11 pmOk nice ! It seems you are using the static library though. Here is the commit as to when I removed it -- but maybe it is time to bring it back.Joerg Oster wrote: ↑Tue Sep 25, 2018 9:24 pm Finally success!
After building the shared library, which took quite some time,
compiling egbbdll worked (with USE_TF = 1).
Only thing I needed to change was line 56 of the egbbdll makefile intoNow I hope the fun part can start!Code: Select all
TF_LIB += $(TF_DIR_LIB)/libtensorflow-core.a
Thanks for your help!
https://github.com/dshawul/egbbdll/comm ... f6fd6ed5e6
The shared library built with --config=monolithic is libtensorflow_cc.so and this is the one that can be used with GPUs.
Also there is an inherent problem with protobuf and tensorflow shared library built outside of bazel.
I was not able to use multiple GPUs with tensorflow_cc and other prebuilt windows libraries as well -- and I spent
literally weeks chasing for a workaround. I couldn't find a solution so i resorted to building using bazel.
Your egbbdll binary will crash because of that where it tries to set the CPU/GPU number, so you need to comment out the following lines
in eval_nn.cppregards,Code: Select all
//std::string dev_name = ((dev_type == GPU) ? "/gpu:" : "/cpu:") + std::to_string(dev_id); //graph::SetDefaultDevice(dev_name, &graph_def);
Daniel
Even if I outcomment the mentioned lines in eval_nn.cpp.
Trying to build with USE_TF = 2 and the makefile (not with Bazel!),
I get an error.
Looking into the makefile, something seems strange:
ifeq ($(USE_TF),1)
TF_DIR=/usr/local
TF_DIR_INC=$(TF_DIR)/include/tensorflow
TF_DIR_LIB=$(TF_DIR)/lib/tensorflow_cc
else ifeq ($(USE_TF),2)
TF_DIR_INC=/home/dabdi/stests/vals/tensorflow <===
TF_DIR_LIB=$(TF_DIR_INC)/bazel-bin/tensorflow
endif
I guess I need to modify TF_DIR_INC to fit my system?
Jörg Oster