Problem using build-script to use gtk-fortran as a dependency

[vmagnin]

Hi,
For some days, I am playing with fpm 0.1.3 alpha on Linux Kubuntu 20.10. I have especially tested the [dependencies] killing feature and it runs smoothly with the helloff.git example in the doc. I am now interested by the build-script variable, which is the true killing feature! I have begun testing it with my gtk-fortran project, which is CMake based. I will now present the details of the steps I have made and the problem:

1. I have created a new fpm project:

$ fpm new gtk_and_fpm
$ fpm build
$ fpm run

2. I have modified the main.f90 as follows, in order to print the exact GTK 3 version available on the machine using directly the functions of GTK:

program main
  use gtk, only: gtk_get_major_version, gtk_get_minor_version, gtk_get_micro_version
  implicit none

  print '(A4,I0,A1,I0,A1,I0)', "GTK ", gtk_get_major_version(),".", &
    & gtk_get_minor_version(), ".", gtk_get_micro_version()
end program main

3. I have first verified that the program is OK by compiling and running it by hand:

~/Fortran/gtk_and_fpm/app$ gfortran main.f90 $(pkg-config --cflags --libs gtk-3-fortran) && ./a.out
GTK 3.24.23

4. I have added those lines in the fpm.toml of my gtk_and_fpm project:

[dependencies]
gtk-3-fortran = { git = "https://github.com/vmagnin/gtk-fortran.git" }

5. I have added an FPM manifest at the root of the gtk-fortran project (gtk3 branch). The modules composing the gtk-fortran library itself is in the src/ directory of the project. And there was already a simple build.sh script there, that basically just do classical things : cd .. ; mkdir build ; cd build ; cmake .. ; make -j :

name = "gtk-3-fortran"
version = "3"
license = "GPLv3"
author = "The gtk-fortran team"
maintainer = "https://github.com/vmagnin/gtk-fortran/issues"
copyright = "2011-2021 The gtk-fortran team"

[library]
build-script="src/build.sh"

6. When typing $ fpm build, the gtk-fortran project is correctly downloaded by git. The building process begins but stops here:

Warning: Return value of function ‘gtk_button_accessible_get_type’ at (1) not set [-Wreturn-type]
build/dependencies/gtk-3-fortran/src/gtk-auto.f90:36089:0:

36089 | function gtk_container_accessible_get_type() bind(c)
      | 
Warning: Return value of function ‘gtk_container_accessible_get_type’ at (1) not set [-Wreturn-type]
 + gfortran -c build/dependencies/gtk-3-fortran/src/gtkenums-auto.f90 -Wall -Wextra -Wimplicit-interface -fPIC -fmax-errors=1 -g -fbounds-check -fcheck-array-temporaries -fbacktrace -fcoarray=single  -J build/gfortran_debug/gtk_and_fpm -I build/gfortran_debug/gtk_and_fpm -o build/gfortran_debug/gtk_and_fpm/build_dependencies_gtk-3-fortran_src_gtkenums-auto.f90.o
f951: Error: Unexpected end of file in ‘build/dependencies/gtk-3-fortran/src/gtkenums-auto.f90’
 Command failed
ERROR STOP 

Error termination. Backtrace:
#0  0x7fc302314d01 in ???
#1  0x7fc302315849 in ???
#2  0x7fc302316f77 in ???
#3  0x55ed6a4dc686 in __fpm_environment_MOD_run
        at src/fpm_environment.f90:127
#4  0x55ed6a4bc15f in build_target
        at src/fpm_backend.f90:240
#5  0x55ed6a4beef2 in __fpm_backend_MOD_build_package
        at src/fpm_backend.f90:74
#6  0x55ed6a4ae050 in __fpm_MOD_cmd_build
        at src/fpm.f90:204
#7  0x55ed6a4a65bf in MAIN__
        at app/main.f90:26
#8  0x55ed6a4a66c7 in main
        at app/main.f90:9

7. Reading those warnings (that does not appear using CMake) and the final error, I suspect that the building is not made by my build.sh script, but directly by fpm. Note also that gtkenums-auto.f90 is not a module but a file that is included in the source code of the gtk-auto.90 module via an INCLUDE statement (that's probably why fpm failed).

8. Therefore, the https://github.com/vmagnin/gtk-fortran/blob/gtk3/fpm.toml file is probably not correct. Can you please guide me to obtain a correct fpm.toml ? I tried some modifications on the local version on my machine but without success.

[LKedward]

Hi Vincent @vmagnin, thanks for reporting. Unfortunately the build-script feature is not yet implemented in the most recent version of fpm (see also #341, #219 for more info) and so yes you're right that fpm is trying to build the project itself.

---

I have been able to build gtk-fortran directly with fpm and run the examples successfully (on Ubuntu) by renaming the included files to src/gtk-auto.inc and src/gtkenums-auto.inc, and adding the necessary link entries to the manifest. However I'm aware that this bypasses the various cmake logic that is used for this project and which I haven't looked into in much detail.

I'm quite excited to be able to use gtk-fortran as an fpm package sometime in the future!

[vmagnin]

Hi Laurence @LKedward !
it seems I am too much "absolutely modern" today... ;-)
Thanks for the referenced issues. I now understand the logic of the problem. I will be patient! It seems so promising...

Great news that you have built gtk-fortran with fpm! I will give a try to your method.

Concerning CMake, it's a powerful tool. Two years ago I made an attempt to migrate to Meson because I feared it will be complicated to adapt CMake to GTK 4, and because GTK itself is now using Meson. But finally it was simple with CMake, and Meson is not yet installing correctly the .mod Fortran files:
mesonbuild/meson#5374

Finally the only thing I don't like with CMake is its verbose syntax. My neurons can't even remember $ cmake -D CMAKE_BUILD_TYPE=debug .., I am always searching in my bash history... I prefer the simple fpm or Meson syntax.

I also read here some discussions about work on fpm detecting automatically the projects using CMake.

[vmagnin]

I report progress and problem using gtk-fortran as a fpm dependency, using fpm 0.5.0 and GFortran 11.2.0 under Ubuntu 21.10:

1. I have created experimental_fpm branches in those two projects:
   https://github.com/vmagnin/gtk-fortran
   https://github.com/vmagnin/gtkzero_fpm
   The last one is using gtk-4-fortran as a dependency and just opens an empty GTK window.

2. In the experimental branch, I have fixed the problem concerning the two Fortran included files by renaming them with the ".inc" extension, as suggested by Laurence @LKedward

3. If I try to build gtkzero_fpm, there is a problem with a gtk-fortran file that I will detail after. Let's remove it with $ rm build/dependencies/gtk-4-fortran/src/mswindowsonly-auto.f90

4. Now fpm builds and runs the project without problem! (I am really impressed by the ability of fpm to do it with so few information in the fpm.toml files...)

Let's now talk about the problem concerning the mswindowsonly-auto.f90 file: in GTK there is some functions whose names are specific to Windows or Unix and in gtk-fortran some are in mswindowsonly-auto.f90 and unixonly-auto.f90. Those two files contain the same module gtk_os_dependent, that's the problem for fpm.

In CMake, we have those lines to deal with it:

if (CMAKE_HOST_WIN32)
  set(sources ${sources} "mswindowsonly-auto.f90")
else()
  set(sources ${sources} "unixonly-auto.f90" "unix-print-auto.f90")
endif()

Maybe I can think about it on the gtk-fortran side (I am not sure these few functions are still useful with nowadays Windows versions; I will investigate).

But on the fpm side, is it possible or will it be possible to deal with that problem ? Or have you any idea to get around ?

[vmagnin]

Concerning the mswindowsonly-auto.f90 and unixonly-auto.f90, the answer I received on the Gnome Discourse is encouraging: the Windows specific file is in fact not necessary. I will work on its deletion.
Concerning unix-print-auto.f90 it won't be a problem as far as its functions are not called by a program under Windows.

But being able to choose the files to compile depending on the OS is probably an expected feature in a package manager.

[ivan-pi]

But on the fpm side, is it possible or will it be possible to deal with that problem ? Or have you any idea to get around ?

There have been discussions in a few places (#611 (comment), #623, #609) about including conditional logic or optional dependencies. Unfortunately, nothing concrete yet.

The only solution I can think of currently is to defer the logic to the preprocessor built in to the compiler. E.g. something like:

! gtk_os_dependent.f90

#if WINDOWS
include "mswindows-auto.inc"
#else
include "unixonly-auto.inc"
include "unixprint-auto.inc"
#endif

Then you should pass a -DWINDOWS when invoking the fpm build command.

[vmagnin]

Thanks @ivan-pi,
so it seems in the air but quite complicated to design and code.
I will try to work on my project without it, and if necessary will have a look on the preprocessor option.

[vmagnin]

I have now fixed the problem in the experimental_fpm branch of gtk-fortran, by deleting those two files which were finally not necessary, as revealed by the GTK developers.
If you clone the experimental_fpm branch of https://github.com/vmagnin/gtkzero_fpm
and do fpm run, it should clone gtk-fortran as a dependency, and launch an empty GTK window. You will need of course to install the GTK4 dev files.
Before closing this issue, I will finalize the work and test it on several platforms.
If some people can test gtkzero_fpm on their machine and post return here, they are welcome!

[vmagnin]

I have a question concerning the [build] link = statement. You may know that GTK is not a library but a collection of library: pkg-config --cflags --libs gtk4 will tell you that you must link -lgtk-4 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -lgdk_pixbuf-2.0 -lcairo-gobject -lcairo -lgraphene-1.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0
Should I put all theses libraries in the [build] link = of the fpm.toml of gtk-fortran? Or should I let the user choose only those that he really needs and put them in the fpm.toml of his project?

For the moment, in the experimental_fpm branches of gtk-fortran and gtkzero_fpm, I have used the second solution. There is no [build] section in gtk-fortran, and in gtkzero_fpm there is only:

[build]
link = [
  "gtk-4",
  "gio-2.0",
  "gobject-2.0"]

It works perfectly with Ubuntu, Fedora and MSYS2/Windows 10.

Can I be sure that the name of these libraries will be the same on all systems? (needed if I put them in the fpm.toml of gtk-fortran).

[vmagnin]

I have now put the link= list in gtk-fortran instead of gtkzero_fpm, with the full list needed to link GTK programs. And I will try it on various systems.

[vmagnin]

It runs fine on Ubuntu 21.10, Fedora 35, FreeBSD 13.0, MSYS2/Windows 10, macOS 10.15.7.

[vmagnin]

The two experimental_fpm branches have been merged into the gtk4 and main branches of the two projects:
https://github.com/vmagnin/gtk-fortran
https://github.com/vmagnin/gtkzero_fpm
That killing feature will be announced with the release of gtk-fortran 4.2 in a few weeks.

But before, in the experimental_fpm branches I will now explore the use of PLplot when gtk-fortran is used as a fpm dependency.

[LKedward]

Hi Vincent @vmagnin, thanks for sharing your progress on this and apologies for the late response! It's really exciting that gtk-fortran is moving towards supporting fpm! I think your experience documented here will be really useful for informing future directions for fpm. I hope to test out your experimental branch soon.

[vmagnin]

I hope to test out your experimental branch soon.

Hi Laurence @LKedward ,
you can already use the gtk4 branch, it's merged. I am interested by feedback.

The experimental_fpm branches will be for experimenting with PLplot+gtk-fortran, starting today.

[vmagnin]

@LKedward
One small defect is that the whole gtk-fortran is git cloned in each project, and it's ~18 Mio. I don't know if it would be possible to download only the useful stuff when using it as a dependency (the root directory and the src/) ?

[ivan-pi]

For such heavy duty libraries, a local dependency might be more suitable. I imagine this similar to Intel MKL oneAPI, i.e. you only want to download the compiler toolchain and SDK's once, and then use pkg-config or some other method to get the right include directories and linkage flags.

Unfortunately it's not entirely clear to me, how would this work together with the current package distribution model.

[vmagnin]

For such heavy duty libraries, a local dependency might be more suitable.

Sure. But a great force of fpm is that it makes so easy to test and discover Fortran projects: with three lines (git clone, cd and fpm run), you can test a Fortran project. Once people have discovered and played with a new library, they could later install it on their system if they use it often.

If you use it in only one, two or three directories, it can probably be acceptable to use it as a dependency, whatever the size.

[ivan-pi]

Indeed, ideally, you'd have a way to do both. I was also thinking about some closed systems (e.g. in an industrial environment or institute) where you can't git clone easily, but you might have several graphical apps sharing the same GTK source folder.

[vmagnin]

By local dependency, did you mean "fpm local dependency"?
Looking in fpm doc, it seems I could git clone one gtk-fortran and put in all my fpm.toml manifests:

[dependencies]
gtk-fortran = { path = "../gtk-fortran" }

instead of cloning it multiple times. Yes, it runs. If we are already in the good branch of gtk-fortran.

[vmagnin]

I have just succeeded running a gtk-fortran PLplot example, by putting in the fpm.toml:

[dependencies]
gtk-fortran = { git = "https://github.com/vmagnin/gtk-fortran.git", branch = "experimental_fpm" }

[build]
link = ["plplot", "plplotfortran"]
external-modules = ["plplot"]

and running with:

$  fpm run --flag '$(pkg-config --cflags --libs plplot plplot-fortran)'

Any comment?

[ivan-pi]

By local dependency, did you mean "fpm local dependency"?

Yes, that's what I meant. It's one way to cut down on git traffic.

Sorry, I'm not familiar enough to comment about the plplot example. In the long run, there was some interest to have built-in pkg-config integration (see #445). I started work on a Fortran interface to libpkgconf a while ago, but never got around to complete it.

[vmagnin]

The gtk-fortran experimental_fpm branch has been merged into gtk4 : PLplot can be used when gtk-fortran is used as a fpm dependency.
Before releasing gtk-fortran 4.2, I must now:

• update doc and tutorials,
• generate new interfaces for GTK 4.6 and GLib 2.72 (Fedora 36),
• and finally update the https://github.com/conda-forge/gtk-4-fortran-feedstock created by @awvwgk

[awvwgk]

and finally update the https://github.com/conda-forge/gtk-4-fortran-feedstock created by @awvwgk

This step is automatic, just create a release and wait for the conda-forge bots to do the work (as long as you keep CMake as primary build system).

[vmagnin]

This step is automatic, just create a release and wait for the conda-forge bots to do the work (as long as you keep CMake as primary build system).

Wow, that's magic @awvwgk ! Yes CMake remains the primary build system. fpm is for the moment only for using gtk-fortran as a fpm dependency.

Maybe we will need to remove https://github.com/conda-forge/gtk-4-fortran-feedstock/blob/main/recipe/cmake.patch
as you had created that new file: cmake/DefaultFlags.cmake

[vmagnin]

This step is automatic, just create a release and wait for the conda-forge bots to do the work (as long as you keep CMake as primary build system).

@awvwgk , typically, how long can it take?
I have pushed a gtk-fortran 4.2.0 release three hours ago: https://github.com/vmagnin/gtk-fortran/releases/tag/v4.2.0

For the moment nothing seemed to happen:
https://github.com/conda-forge/gtk-4-fortran-feedstock

And I read:

Updating gtk-4-fortran-feedstock
If you would like to improve the gtk-4-fortran recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build.