Compiling for Android revisited

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
abik
Posts: 740
Joined: Fri Dec 01, 2006 9:46 pm
Location: Mountain View, CA, USA
Full name: Aart Bik
Contact:

Compiling for Android revisited

Post by abik » Thu Nov 20, 2014 12:46 am

If you are already running Android 5.0 (Lollipop), you may have noticed issues with some Android engines (either provided as direct binary, such as these examples, or provided in a directly-available form such as this chess engine package). The underlying problem is that non-PIC/non-PIE code is no longer supported for security reasons, which implies that prior dynamically compiled engines no longer work.

I did a quick experiment with my own engine, compiled different ways, and on 5.0, dynamically compiled binaries only run when made position independent. Static compiled binaries still work. Summarizing:

Code: Select all

bikjump18_dyn      : runs prior to Lollipop, but not on Lollipop
bikjump18_dyn_pie  : runs on Lollipop (but only Android 4.1+)
bikjump18_static   : runs on all
Although the recommendation going forward is to use position-independent compilations, at the moment providing a static binary seems the simplest solution.

I hope to update my own engine package soon so that all works well again.

Ferdy
Posts: 4074
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Compiling for Android revisited

Post by Ferdy » Thu Nov 20, 2014 3:33 pm

abik wrote:If you are already running Android 5.0 (Lollipop), you may have noticed issues with some Android engines (either provided as direct binary, such as these examples, or provided in a directly-available form such as this chess engine package). The underlying problem is that non-PIC/non-PIE code is no longer supported for security reasons, which implies that prior dynamically compiled engines no longer work.

I did a quick experiment with my own engine, compiled different ways, and on 5.0, dynamically compiled binaries only run when made position independent. Static compiled binaries still work. Summarizing:

Code: Select all

bikjump18_dyn      : runs prior to Lollipop, but not on Lollipop
bikjump18_dyn_pie  : runs on Lollipop (but only Android 4.1+)
bikjump18_static   : runs on all
Although the recommendation going forward is to use position-independent compilations, at the moment providing a static binary seems the simplest solution.

I hope to update my own engine package soon so that all works well again.
Thanks for this info. I have not tried this Android 5 yet, but I compiled Deuterium using Sourcery Codebench, a guide from Pedro, based from here.
http://talkchess.com/forum/viewtopic.ph ... 93&t=51660
Is this what you called static?

BTW how is Chess for Android now? I hope to see an adjudication option in the tournament, perhaps the TC too. Very nice app.

User avatar
abik
Posts: 740
Joined: Fri Dec 01, 2006 9:46 pm
Location: Mountain View, CA, USA
Full name: Aart Bik
Contact:

Re: Compiling for Android revisited

Post by abik » Thu Nov 20, 2014 5:51 pm

Ferdy wrote:Is this what you called static?
You will have to use the -static flag to prevent linking shared libraries (and as a result, this yields a slightly bigger binary). If you have a binary available I am happy to test this on my Android 5.0.
Ferdy wrote:BTW how is Chess for Android now? I hope to see an adjudication option in the tournament, perhaps the TC too. Very nice app.
I am very excited about the upcoming release. A complete rewrite of engine support under the hood, clocks, multiple analysis lines during tournaments. As for adjudication, right now the official rules are used (insufficient material etc.), it will be a bit harder to go beyond that.

mar
Posts: 1992
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Compiling for Android revisited

Post by mar » Thu Nov 20, 2014 7:15 pm

Hmm... If I understand correctly PIC/PIE is the same as ASLR on Windows.
Shared libraries must use some sort of relocation sheme already even on previous OSes,
so I see no reason to segfault on the old ones :shock:
They could've used a simple stub and pack relocations in some r/o section,
determining base address on the fly.

Oh well...

User avatar
abik
Posts: 740
Joined: Fri Dec 01, 2006 9:46 pm
Location: Mountain View, CA, USA
Full name: Aart Bik
Contact:

Re: Compiling for Android revisited

Post by abik » Thu Nov 20, 2014 7:32 pm

mar wrote:so I see no reason to segfault on the old ones :shock:
Well, not sure if it matters much, but it is not a segfault. Support was simply dropped altogether for security reasons.

Code: Select all

$ ./bikjump18_dyn
error: only position independent executables (PIE) are supported.

Ferdy
Posts: 4074
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Compiling for Android revisited

Post by Ferdy » Fri Nov 21, 2014 2:07 am

Aart Bik wrote:You will have to use the -static flag to prevent linking shared libraries (and as a result, this yields a slightly bigger binary). If you have a binary available I am happy to test this on my Android 5.0.
Here it is, the file is a little over 1mb. It works in Android 4.4.2, on Chess for android, analyze this, droidfish, pgn master and scid on the go gui's.

http://www.mediafire.com/download/k95ya ... 34_129.rar

mar
Posts: 1992
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Compiling for Android revisited

Post by mar » Fri Nov 21, 2014 5:35 am

I haven't tried myself but this https://groups.google.com/forum/#!msg/a ... KU32Tr99oJ suggests they get segfaults with PIE on pre-4.1.
EDIT: actually, what I wrote earlier probably wouldn't work anyway because I guess code section resides in r/o pages anyway, so forget what I wrote :)

User avatar
abik
Posts: 740
Joined: Fri Dec 01, 2006 9:46 pm
Location: Mountain View, CA, USA
Full name: Aart Bik
Contact:

Re: Compiling for Android revisited

Post by abik » Fri Nov 21, 2014 6:09 am

mar wrote:they get segfaults with PIE on pre-4.1.
Ah, now I see what you meant by "the old ones" (I thought you meant the old binaries, but you meant the older version). So I was talking on non-PIE behavior on 5.0, but you were talking about PIE behavior pre-4.1. Sorry about not reading more carefully.

User avatar
abik
Posts: 740
Joined: Fri Dec 01, 2006 9:46 pm
Location: Mountain View, CA, USA
Full name: Aart Bik
Contact:

Re: Compiling for Android revisited

Post by abik » Fri Nov 21, 2014 5:29 pm

Ferdy wrote:Here it is, the file is a little over 1mb.
Works great on 5.0. No worries.

Code: Select all

$ ./Deuterium-v14_3_34_129                          
info string Deuterium v14.3.34.129 32bit android
uci
id name Deuterium v14.3
id author Ferdinand Mosca
option name Hash type spin default 32 min 16 max 2048
option name EvalHash type spin default 8 min 8 max 32
option name PawnHash type spin default 2 min 1 max 8
option name KingAndPawnHash type spin default 2 min 1 max 8
option name Ponder type check default true
option name KingSafetyPercent type spin default 100 min 0 max 1000
option name MidGameMobilityPercent type spin default 100 min 0 max 1000
option name EndGameMobilityPercent type spin default 100 min 0 max 1000
option name OffensivePercent type spin default 100 min 100 max 1000
option name DefensivePercent type spin default 100 min 100 max 1000
option name FutilityPruning type check default true
option name LateMoveReduction type check default true
option name NullMove type check default true
option name Razoring type check default true
option name GoodEvaluationPruning type check default true
option name RepetitionScore type spin default 0 min -500 max 500
option name InsufficientMaterialScore type spin default 0 min -500 max 500
option name FiftyMoveScore type spin default 0 min -500 max 500
option name StaleMateScore type spin default 0 min -30000 max 30000
option name TimeBufferMilliSec type spin default 60 min 0 max 1000
option name CheckExtension type check default true
option name EndingExtension type check default true
option name RecaptureExtension type check default true
option name PawnTo7thRankExtension type check default true
option name MultiPV type spin default 1 min 1 max 300
option name Engine_LimitStrength type check default false
option name Engine_Elo type spin default 2500 min 1000 max 2500
option name UCI_AnalyseMode type check default false
uciok

Ferdy
Posts: 4074
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Compiling for Android revisited

Post by Ferdy » Sat Nov 22, 2014 3:13 am

abik wrote:
Ferdy wrote:Here it is, the file is a little over 1mb.
Works great on 5.0. No worries.
Alright thanks :) .

Post Reply