go get for go-python3 on mac M1 is not working.

[avishayzamir]

I am having a new Mac with an M1 processor installed.

I installed a Golang version with an arm version for Mac M1 from the official site: https://go.dev/dl/
I downloaded Python3 for M1.
After that, I tried to run: go get github.com/DataDog/go-python3
and got an error message attached to the file.
I tried also to run: go get github.com/go-python/cpy3
and I got the same error.
Untitled.txt

On the other hand, I downloaded and installed the Golang for the x86 processor, it works fine.
but, I have to work with the right version on my Mac (Golang version for M1, arm version).

[christian-korneck]

as far as I can see you're trying to use a darwin_amd64 (Intel) Python with a darwin_arm64 (M1) Go. (Unfortunately there are no official Python 3.7 darwin_arm64 builds, so that's no surprise). A quick fix is to use a darwin_amd64 (Intel) variant of Golang (go tool) with Rosetta2 on your M1 Mac. This is due to CGO needing to link against the Python dylib.

We've also discussed this problem here:
DataDog#38 (comment)

(JFWIW: There's work going on to support more recent Python versions with go-python/cpy3, for which there are darwin_arm64 builds).

[avishayzamir]

Using a darwin_amd64 (Intel) variant of Golang is not possible, because in that case debugging the code on my Mac (m1) is not possible.

[christian-korneck]

what exactly isn't working?

[avishayzamir]

I can't run my code in debug mode. As a developer, I need it

[christian-korneck]

Do you have an example? (I imagine you're trying to use GDB or Delve and you get an error?). Just trying to understand.

[avishayzamir]

Sure, I need to re-install Go and Python3 and will attach a screenshot

[avishayzamir]

Golang	GoLand	Debug	Go-Python/cpy3
go1.17.7.darwin-amd64.pkg (Intel)	Intel	could not launch process: can not run under Rosetta, check that the installed build of Go is right for your CPU architecture	succed
go1.17.7.darwin-amd64.pkg (Intel)	Apple	Debugging programs compiled with go version go1.17.7 darwin/amd64 is not supported. Use go sdk for darwin/arm64	succed
go1.17.7.darwin-arm64.pkg (Apple)	Intel	Debugging programs compiled with go version go1.17.7 darwin/amd64 is not supported. Use go sdk for darwin/arm64	failed
go1.17.7.darwin-arm64.pkg (Apple)	Apple	succed	failed

[avishayzamir]

Hi Christian,
I attached the table above.

Thank you

[shayneoneill]

Yeah the amd64 option is a non starter , it really does need to be arm.

[shayneoneill]

Just so folks know. 3.7.16 DOES compile on a mac M1. Earliier ones dont I think.

You'll have to compile it yourself and you MIGHT have troubles with ctypes and ffi. Still havent figured out yet how to tell go-python which install of python to use as pkg-config points to the wrong python, and I cant find any flags to pass go to tell it to use an alternative one.

[shayneoneill]

Ok. Got it working. Heres the deal.
Go get the SOURCE for 3.7.16 and compile it. You may want to look around for how to get the prereqs in (OpenSSL etc.)

./configure
make
make install 

It MIGHT complain about ffi. and throw errors. Thats a tough one. In the python 3.7 and earlier lines python included a vendored version of libffi that seems to explode in flames on the macm1. I havent really solved this one yet. Theres a --use-system-ffi option for config, but it kind of doesnt work. There are some patches for later versions of pythons build scripts that could be backported but I havent quite got these to work yet. You CAN still install this python , but ctypes will be broken which might break some of the math packages. Work in process.

Now Make type "which pkg-config" to find out where your pkg-config app is installed. If its in a homebrew opt, it likely will point to the wrong version of python. You'll wanna get the python3.pc and python3.7.pc files from /usr/local/lib/pkgconfig and copy these into the homebrew equivelent lib/pkgconfig

You'll know its working when;- pkg-config python3 --cflags reports -I/usr/local/include/python3.7m

THEN you can compile and run it.

This still isn't ideal, without ctypes and ffi, a lot of pythons signature math/science packages are unlikely to work. So at this stage the game is still about getting python3 to work with FFI and thus CTYPES. Or better still to actually support a modern version of python3 as 3.7 is solidly deprecated.

edit: Apparently this works with 3.8 with a slight modification to the library to remove a deprecated function.