64 bit support

Forum for discussing getting Ecere running on other platforms (iPhone, FreeBSD, SunOS, consoles...) and improving support on existing platforms (e.g. Mac OS X)
jerome
Site Admin
Posts: 608
Joined: Sat Jan 16, 2010 11:16 pm

64 bit support

Post by jerome »

So we've been wanting to run in 64 bit for the longest time...

At first I did not have a 64 bit PC so I had a good excuse, but those times are long gone...
We still have the excuse that MinGW32 does not support 64 bit, but we have MinGW-W64 to try :)

It seems that it'd be easier to get this started on Linux, however, and there's already some basic awareness in the IDE to build in 32 vs 64 bit. The options are still missing a GUI in Project settings however (contributions welcome to add it - http://ecere.com/mantis/view.php?id=743), so we'll have to hack at the project files with a text editor for now...

vendu mentionned the interest to get this 64 bit support in eC going, so here's why I'm bothering with this at all :)

The problem of 64 bit lies in a few areas...
The eC compiler will need some awareness of the size of data types, and so will the eC runtime.
I'm almost certain some generated C code makes some bad assumptions, and coded code in the runtime and/or compiler as well... Note that ideally a distinction should be made between host and target, so a 64 bit eC compiler should be able to compile eC code meant to run on a 32 bit machine... Ideally, there won't be too much attention that must be given to this, and generated C code should be able to build on both 32 and 64 bit systems (In fact, this is a requirement for the bootstrapping compiler at the minimum, which is generated C code, to be able to run on both 32 and 64 bit!)

So things to watch for: sticking void * in int (I've done that a lot).
e.g. in Window::id, DataRow::tag, etc. :)
The new collection classes (containers) were thought with 64 bit in mind, but there might still be some issues with them.
The old containers however, which are still used in many areas (especially throughout the compiling tools), must be making a lot of sizeof(int) == sizeof(void *) assumptions. We need to phase them out completely, in favor of the new containers, but for now it might be easier to just fix them to work in both 32 and 64 bit. These are:
- OldArray
- OldList (Along with related classes NamedItem, NamedLink, Item, OldLink)
- BinaryTree (Along with BTNode)

Multiple-data types unions and such are used, and we'll have to be careful about the offset (little-endian vs big-endian 64 bit machines!)


So... here's how I propose to start. First we should try to get a very basic hello world application compiled and running in 64 bit.

Code: Select all

class Hello : Application
{
   void Main()
   {
      PrintLn("Hello, 64 bit!!");
   }
}
This will necessitate ecereCOM to be built in 64 bit (You'll also have to change 'ecere' to 'ecereCOM' in the Project Settings/Linker tab)

First of all, checkout the dev branch from github @ https://github.com/ecere/sdk/tree/dev .

make yourself some work folder, say /ecere64/
And in there make a subfolder /ecere64/ecereCOM
Copy sdk/ecere/src and sdk/ecere/ecereCOM.epj in there .

Then make a new empty project in /ecere64/hello/ and add the above snippet in to a file called hello.ec

Now we'll text edit these .epj to make them build in 64 bit. We need to add the option "BuildBitDepth" : "Bits64" under the top level options, so Hello.epj will look like this

Code: Select all

{
   "Version" : 0.2,
   "ModuleName" : "Hello",
   "Options" : {
      "Warnings" : "All",
      "TargetType" : "Executable",
      "TargetFileName" : "Hello",
      "Libraries" : [
         "ecere"
      ],
      "BuildBitDepth" : "Bits64"
   },
   "Configurations" : [
      {
         "Name" : "Debug",
         "Options" : {
            "Debug" : true,
            "Optimization" : "None",
            "PreprocessorDefinitions" : [
               "_DEBUG"
            ]
         }
      },
      {
         "Name" : "Release",
         "Options" : {
            "Debug" : false,
            "Optimization" : "Speed"
         }
      }
   ],
   "Files" : [
      "hello.ec"
   ],
   "ResourcesPath" : "",
   "Resources" : [

   ]
}
Now reopen the project and try to build it... (I think it will actually be loading the 32bit version of libecereCOM.so to do the eC compiling, since the installed eC compiler is built in 32 bit... This might be fine, if we manage to produce C code that can then be compiled either or 32 or 64... We just have to be careful about any structure size computation that might get messed up, though if it's all done at runtime we should be fine).

To link and/or run this Hello application however, we'll need to build ecereCOM in 64 bit as well.

So do a Project/Add project to workspace and add /ecere64/ecereCOM.epj , which you text-edited to build as 64 bit as well. Try to build it, and see how far you get :)

Once this hello world is going, we can try to build the full ecere runtime, and see how far we get with a GUI application =)

Let's keep this thread going till we can do 64 bit =)

-Jerome
D.Bane
Posts: 78
Joined: Thu Jul 29, 2010 2:51 pm

Re: 64 bit support

Post by D.Bane »

Hi Jerome,

this is certainly a good news.

I have checked the Mantis link.. Would it not be better to instead of
Platform::win32 to Platform::windows
do a

Code: Select all

Platform::win32 and Platform::win64
which would allow the programmer to check in his/hers application if one code or another should be run.

Also Jerome when compiling the 64bit app, the int class would still be 32 bit in the 64bit application, and the int64 would be of the same size as well, am I right?

Will start with the compilation at the evening (hopefully) and will let you know :) (<- I need to update the ecere as well)

Best Regards,
D.Bane
No army is to big, if the heart is beating strong.
There's no bad luck, it's just easier to blame someone else.
jerome
Site Admin
Posts: 608
Joined: Sat Jan 16, 2010 11:16 pm

Re: 64 bit support

Post by jerome »

Hi D.Bane!

Well the idea is that the platform is Windows, or Linux...
And the bit depth can be checked independently...
(Although ideally, the application would not need to know about that... The only major difference being the size of pointers).

Yes, that's correct. int/uint will always be 32-bit in eC, and int64/uint64 will always be 64 bit.

Regards,

Jerome
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

Hi all!

Is anyone actively working on this?

Can we work together to get this done?

https://github.com/redj/sdk/tree/bit64

-redj
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

here's the output of building hello64.epj so far...

Code: Select all

Default Compiler
Rebuilding project hello64 using the Debug configuration...
Target and object files deleted
Generating symbols...
hello64.ec
Compiling...
hello64.ec
hello64.c
Writing symbol loader...
hello64.main.ec
hello64.main.ec
hello64.main.c
Linking...
   Linker Error: skipping incompatible /usr/lib/ec/libecereCOM.so when searching for -lecereCOM
   Linker Error: cannot find -lecereCOM
   collect2: ld returned 1 exit status

hello64 (Debug) - 2 errors, no warning
we need to compile ecereCOM in 64 bit (see next post) first.

-redj
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

now the output of building ecereCOM.epj in 64 bit...

Code: Select all

Default Compiler
Rebuilding project ecereCOM using the Debug configuration...
Target and object files deleted
Generating symbols...
Mutex.ec
i18n.ec
Array.ec
AVLTree.ec
BuiltInContainer.ec
Container.ec
CustomAVLTree.ec
LinkList.ec
List.ec
Map.ec
BinaryTree.ec
BTNode.ec
dataTypes.ec
instance.ec
String.ec
OldList.ec
Compiling...
Mutex.ec
Mutex.c
i18n.ec
i18n.c
Array.ec
Array.c
AVLTree.ec
AVLTree.c
BuiltInContainer.ec
BuiltInContainer.c
Container.ec
Container.c
CustomAVLTree.ec
CustomAVLTree.c
LinkList.ec
LinkList.c
List.ec
List.c
Map.ec
Map.c
BinaryTree.ec
BinaryTree.c
BTNode.ec
BTNode.c
dataTypes.ec
dataTypes.c
instance.ec
instance.c
String.ec
String.c
OldList.ec
OldList.c
instance.c
Writing symbol loader...
ecereCOM.main.ec
ecereCOM.main.ec
ecereCOM.main.c
Linking...
   ../ecere/obj/ecereCOM.debug.linux/Mutex.o: In function `__ecereConstructor___ecereNameSpace__ecere__sys__Mutex':
   ../../ecere64/ecere/src/sys/Mutex.ec:56: error:undefined reference to `pthread_mutexattr_init'
   ../../ecere64/ecere/src/sys/Mutex.ec:62: error:undefined reference to `pthread_mutexattr_settype'
   ../../ecere64/ecere/src/sys/Mutex.ec:66: error:undefined reference to `pthread_mutexattr_destroy'
   ../ecere/obj/ecereCOM.debug.linux/instance.c.o: In function `Instance_Module_Load':
   ../../ecere64/ecere/src/com/instance.c:306: error:undefined reference to `dlopen'
   ../../ecere64/ecere/src/com/instance.c:318: error:undefined reference to `dlopen'
   ../../ecere64/ecere/src/com/instance.c:323: error:undefined reference to `dlsym'
   ../../ecere64/ecere/src/com/instance.c:324: error:undefined reference to `dlsym'
   ../../ecere64/ecere/src/com/instance.c:326: error:undefined reference to `dlclose'
   ../ecere/obj/ecereCOM.debug.linux/instance.c.o: In function `Instance_Module_Free':
   ../../ecere64/ecere/src/com/instance.c:371: error:undefined reference to `dlclose'
   ../ecere/collect2: ld returned 1 exit status

ecereCOM (Debug) - 9 errors, no warning
compiling seems to work well enough even if the code is probably buggy but the immediate problem is linking.

the command:

Code: Select all

gcc  -L/usr/lib/ec -m64   -Wl,--no-undefined obj/ecereCOM.debug.linux/Mutex.o obj/ecereCOM.debug.linux/i18n.o obj/ecereCOM.debug.linux/Array.o obj/ecereCOM.debug.linux/AVLTree.o obj/ecereCOM.debug.linux/BuiltInContainer.o obj/ecereCOM.debug.linux/Container.o obj/ecereCOM.debug.linux/CustomAVLTree.o obj/ecereCOM.debug.linux/LinkList.o obj/ecereCOM.debug.linux/List.o obj/ecereCOM.debug.linux/Map.o obj/ecereCOM.debug.linux/BinaryTree.o obj/ecereCOM.debug.linux/BTNode.o obj/ecereCOM.debug.linux/dataTypes.o obj/ecereCOM.debug.linux/instance.o obj/ecereCOM.debug.linux/String.o obj/ecereCOM.debug.linux/OldList.o obj/ecereCOM.debug.linux/instance.c.o obj/ecereCOM.debug.linux/ecereCOM.main.o  -shared   -lm -o obj/ecereCOM.debug.linux/libecereCOM.so
it's output:

Code: Select all

Linking...
   ../ecere/obj/ecereCOM.debug.linux/Mutex.o: In function `__ecereConstructor___ecereNameSpace__ecere__sys__Mutex':
   ../../ecere64/ecere/src/sys/Mutex.ec:56: error:undefined reference to `pthread_mutexattr_init'
   ../../ecere64/ecere/src/sys/Mutex.ec:62: error:undefined reference to `pthread_mutexattr_settype'
   ../../ecere64/ecere/src/sys/Mutex.ec:66: error:undefined reference to `pthread_mutexattr_destroy'
   ../ecere/obj/ecereCOM.debug.linux/instance.c.o: In function `Instance_Module_Load':
   ../../ecere64/ecere/src/com/instance.c:306: error:undefined reference to `dlopen'
   ../../ecere64/ecere/src/com/instance.c:318: error:undefined reference to `dlopen'
   ../../ecere64/ecere/src/com/instance.c:323: error:undefined reference to `dlsym'
   ../../ecere64/ecere/src/com/instance.c:324: error:undefined reference to `dlsym'
   ../../ecere64/ecere/src/com/instance.c:326: error:undefined reference to `dlclose'
   ../ecere/obj/ecereCOM.debug.linux/instance.c.o: In function `Instance_Module_Free':
   ../../ecere64/ecere/src/com/instance.c:371: error:undefined reference to `dlclose'
   ../ecere/collect2: ld returned 1 exit status

ecereCOM (Debug) - 9 errors, no warning
it's possible to compile this successfully if we add dl and pthread to the linux libraries in project settings. but the thing is, ecereCOM is supposed to have the m library as it's only dependency. so I guess some #ifdef(s) must be wrong for 64 bit compilation or something else is going on...

any thoughts?

-redj
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

say we charge ahead having added dl and pthread to bit64/ecere/ecereCOM.epj...

adding ../ecere/obj/ecereCOM.release.linux to the additional library dirs of bit64/hello64/hellp64.epj we can now link hello64 successfully:

Code: Select all

Default Compiler
Relinking project hello64 using the Debug configuration...
Linking...
   Linker Error: skipping incompatible /usr/lib/ec/libecereCOM.so when searching for -lecereCOM

hello64 (Debug) - 1 error, no warning
it says 1 error but the linker only skipped the installed 32 bit libecereCOM.so when linking and found our newly compiled 64 bit libecereCOM.so. we can now try running hello64! \o/

-redj
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

now running hello64 in debug mode...

not surprisingly:

Code: Select all

Signal received: SIGSEGV - Segmentation fault
    (null):0
and our call stack:

Code: Select all

  0 inside ??, /lib/libc.so.6
  1 __ecereNameSpace__ecere__sys__BinaryTree_CompareString Method, src/com/BinaryTree.ec:33
  2 __ecereNameSpace__ecere__sys__BTNode_Find Method, src/com/BTNode.ec:207
  3 __ecereNameSpace__ecere__sys__BinaryTree_Find Method, src/com/BinaryTree.ec:70
  4 __ecereNameSpace__ecere__com__ScanNameSpace Function, src/com/instance.ec:2852
  5 __ecereNameSpace__ecere__com__SearchNameSpace Function, src/com/instance.ec:2892
  6 __ecereNameSpace__ecere__com__eSystem_FindClass Function, src/com/instance.ec:3041
  7 __ecereNameSpace__ecere__com__eSystem_RegisterClass Function, src/com/instance.ec:2273
  8 __ecereNameSpace__ecere__com__LoadCOM Function, src/com/instance.ec:5782
  9 __ecereNameSpace__ecere__com____ecere_COM_Initialize Function, src/com/instance.ec:5972
 10 main Function, obj/debug.linux/hello64.main.ec:32
looks like a BinaryTree issue...

could we please have some pointers on how to fix our good friends BinaryTree and BTNode?

could the issue at hand be this code:

Code: Select all

int (*CompareKey)(BinaryTree tree, uint a, uint b);
it's using uint to pass data/pointers but a 64 bit pointer won't fit in uint?

this could be more complex and involve a compiler issue...

who's going to help?

the latest code is at https://github.com/redj/sdk/tree/bit64

I'm leaving the bit64 branch as it is for now...

please help me keep this going!

-redj
redj
Posts: 105
Joined: Sun Jan 17, 2010 10:03 am

Re: 64 bit support

Post by redj »

oh, regarding editing project files manually to add the building in 64 bit (or 32 bit) option:

Code: Select all

      "BuildBitDepth" : "Bits64"
I have added the gui to do this using a checkbox in the Project Setting dialog of the IDE.

see the commit.

-redj
jerome
Site Admin
Posts: 608
Joined: Sat Jan 16, 2010 11:16 pm

Re: 64 bit support

Post by jerome »

it's possible to compile this successfully if we add dl and pthread to the linux libraries in project settings. but the thing is, ecereCOM is supposed to have the m library as it's only dependency. so I guess some #ifdef(s) must be wrong for 64 bit compilation or something else is going on...
In fact pthread and dl should probably be there, they must somehow just be picked up automatically in 32 bit or something...

- pthread is used for mutexex which is used by the eC memory manager (included in ecereCOM)
- dl is used for loading eC shared library modules (dlopen)

You'll also notice they are already in the Release config, and in Makefile.ecereCOM (Should probably be moved to Common).
Post Reply