[Gambas-devel] gbmake. A drop-in replacement to automake

[Sebastian Kulesz]

On Thu, Sep 6, 2012 at 12:34 AM, Benoît Minisini
<gambas at ...1...> wrote:
> Le 06/09/2012 05:19, Sebastian Kulesz a écrit :
>> On Wed, Sep 5, 2012 at 4:01 PM, Benoît Minisini
>> <gambas at ...1...> wrote:
>>> Le 05/09/2012 13:43, Benoît Minisini a écrit :
>>>> Le 05/09/2012 05:02, Sebastian Kulesz a écrit :
>>>>> Hi all !!
>>>>>
>>>>> I really think Gambas is a great language to work with, but lets face
>>>>> it, from the point of view of a beginner, distributing an application
>>>>> is a pain in the a**. There is no *standard* way to build a project
>>>>> (other than executing gbc3/gba3 directly if we let the IDE aside),
>>>>> this makes it difficult to create debian packages, PPAs, etc. as they
>>>>> all require some sort of autoconf script to manage the process.
>>>>> Without the IDE, an user can do much. And with it, this implies
>>>>> installing every component it depends on (a total of 15).
>>>>>
>>>>> To create a PPA of a gambas application, for example, the user has 2
>>>>> choices. He can either write a Makefile (more bugs, no portability,
>>>>> etc) or create a dummy autoconf package, skim the files and replace
>>>>> what is needed (package name, version, etc), and merge it with his
>>>>> project.
>>>>>
>>>>> These are really just a few examples of the problems i faced not so
>>>>> long ago. With this in mind, i created this little tool to automate
>>>>> the building of gambas apps. It works as a *drop in* replacement for
>>>>> make (*seriously*, you can run "make", "make clean", and "sudo make
>>>>> install" without a single script, autoconf, automake, m4 macro).
>>>>>
>>>>> I also wrote the app with Quickly [0] in mind. For example, if you
>>>>> want to debianize a project, you can run "gbmake debianize" or just
>>>>> "make debianize" and that is it. It will automatically create a
>>>>> skeleton debian/ folder which should work out-of-the-box. Maybe a
>>>>> "make source" to create a source archive?
>>>>>
>>>>> I just started it today, so only the building process is really
>>>>> implemented (build, clean, install) but I really see a lot of
>>>>> potential here. Once it's completed, the autotools package may be
>>>>> dropped (it will be completely replaced and the user will have to do
>>>>> nothing). Uploading packages to a PPA will be much easier. Users won't
>>>>> depend on the IDE to perform administrative tasks, such as building a
>>>>> package or a source archive.
>>>>>
>>>>> I will need some help to port the packaging code in the IDE to work in
>>>>> headless mode. I'm trying to make this tool to only depend on the
>>>>> interpreter, so no single component has to be installed for it to
>>>>> work. This means implementing a custom option parser, and settings
>>>>> parser. I have zero knowledge of Slackware, OpenSUSE, Mandriva, Mageia
>>>>> and Fedora (and it's derivatives), so i will only be able to port the
>>>>> code, but i can't test for any bugs or regressions.
>>>>>
>>>>> How does it works? It's really simple actually. You will see 2
>>>>> Makefiles included. the one named Makefile.1 should be used to install
>>>>> the app in the first place.
>>>>>
>>>>> sudo make -f Makefile.1 install
>>>>>
>>>>> This command will build the project and install it (/usr/bin/gbmake).
>>>>> The second Makefile only wraps the commands sent to "make" to
>>>>> "gbmake". It's really the same, but we are all used to type make *,
>>>>> aren't we? ;-)
>>>>>
>>>>> In this case, only adding this Makefile to a project will let "make"
>>>>> work out of the box, otherwise, "gbmake" will work just fine.
>>>>>
>>>>> I will try to have the Debian part finished by the end of the week so
>>>>> i can provide a working example, this is just a *proof of concept*,
>>>>> and a call for everybody who would like to help. I'm a little skeptic
>>>>> about merging this to trunk (even as a branch), as it's a work in
>>>>> progress, and it will just confuse the users. Besides, 3.3 is around
>>>>> the corner and this will only bring more bugs. Maybe 3.4 if we hurry.
>>>>>
>>>>> I'm really anxious to hear your comments!
>>>>>
>>>>> Thanks!
>>>>>
>>>>> [0] https://wiki.ubuntu.com/Quickly
>>>>>
>>>>
>>>> Good idea, but the problem is code duplication between "gbmake" and the IDE.
>>>>
>>>> I suggest taking the code from the IDE instead of rewriting your own (or
>>>> mix them if yours is better).
>>>>
>>>
>>> Yep. You tried to rewrite things already done by the IDE, but you didn't
>>> do it correctly.
>>>
>>> You should make a list of all project management features you need (how
>>> to clean a project, how to compile it...), and I will centralize all
>>> these features into an IDE module that will be able to be shared by the
>>> gambas make utility.
>>>
>>> Regards,
>>>
>>> --
>>> Benoît Minisini
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Gambas-devel mailing list
>>> Gambas-devel at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/gambas-devel
>>
>> For the moment there *will* be code duplication, but in the end, using
>> gbmake over the available implementations will require much less code.
>
> I think you don't understand : I don't want code duplication to save
> lines of code. I don't want code duplication because you are doing
> things incorrectly in gbmake. For example cleaning the project.
>
> As the IDE packager won't disappear, we must find a way to *not*
> duplicate features like that, otherwise it will be become a nightmare to
> maintain.
>
> So I proposed that all process that depends on the structure of the
> project directory (which is *maintained* by the IDE) is put in a
> specific module. Then that module will be shared between the IDE and the
> gbmake project through a symbolic link.

Wait! I thought you were talking about an exported module, but a
symlink is way better than what i had in mind.

>
>>
>> A few examples:
>>
>> I just finished the archlinux packaging module. If you take a look at
>> the code the IDE uses to package it (more than 200 loc only to
>> package, leaving dependency checks out), you will see a lot of
>> "install" commands, conditionals that check what type of project we
>> are dealing with, etc.
>> With gbmake, we only need to execute 2 commands ("make" and "make
>> install"). gbmake will itself check for dependencies (the package
>> manager should do it, but if installed from source they have to be
>> ignored), determine what type of project it is, and install the
>> required files (this last part is not yet implemented, only basic
>> installation")
>> That 200 loc where reduced to just (fully abstracted) 80, *including
>> dependency checks*.
>>
>> I'm planning the same with the debian part. There are more than 200
>> loc *only* to build a deb package. gbmake can do that by just
>> "debianizing" a temporary folder ("gbmake debianize" will customize
>> the files under debian/* using data prom the proejct configuration)
>> and running "debuild -b", even from a terminal.
>>
>> I don't think gbmake should depend on a module inside the IDE, because
>> we are missing the fact that it can run without a gui.
>
> This has no relation at all with GUI or not, and there is no dependency,
> as we are sharing only source code! Once gbmake is compiled, it has no
> dependency anymore on the IDE.
>
> But then we are sure that when gbmake wants to clean a project, it does
> it the good way. And if the cleaning process changes, the gbmake source
> code is automatically updated.

Same as above.

I'm not exactly sure how we should create this "shared" module(s).
It's really hard to debug as some functions depend on an enormous tree
of classes that would also have to be shared (dependency checking,
project information, etc) and that can't just be symlinked inside
gbmake. How are you planing to achieve this?

>
>> Besides, i'm
>> trying to abstract specific code into separate modules, so that
>> functions that deal with debian packages, for example, resides in a
>> separate module that only gets called when needed (easier to code,
>> easier to maintain).
>>
>>> I don't understand why you need two Makefiles. I imagine that the IDE
>>> would automatically generate a Makefile file inside the source directory
>>> with a configure script to detect that Gambas is installed. That way,
>>> you just have to uncompress the Gambas source archive, run
>>> './configure', 'make' and 'make install', and everything works as
>>> expected. Is it what you have in mind?
>>
>> Makefile.1 was only included so that you can install the tool without
>> much trouble. The other Makefile is a generic one, this should be put
>> inside every project if you wish to run gbmake trough make, but that's
>> optional. The configure script should *only* check if the gambas
>> interpreter is installed, everything else is done automatically.
>
> Again, I don't understand what you have in mind.
>
> The configure script normally must check every dependency of the program
> that is going to be compiled and installed. 'make' does the compilation,
> and 'make install' does the installation.
>
> And I guess that standard behaviour is expected by the Ubuntu automatic
> compilation and packaging process?

You are right! I'm thinking of a configure script that performs the
process in two steps. First it checks if the interpreter is installed,
second, if the first step succeeded, call gbmake to check
dependencies. It already does that internally, but i will add a
special call to perform this checks. What do you think of this?

>
>>
>> As i said on the first mail, that was just a proof of concept. I'm
>> rewriting most of the packaging code (while still adding some missing
>> features), but still using some parts of it. It will take some time to
>> get it to work stable enough to be merged.
>>
>> I will send another source package once i finish the debian module, so
>> those who want can play with it.
>
> As for letting gbmake creating binary packages, ok. But again, the
> packaging code in the IDE is not dependent on the GUI, and I don't
> understand why you want to rewrite it, as AFAIK, it works. Why not just
> sharing it and using it?

I'm not rewriting every bit of it, i'm just adapting and expanding
where i see some missing features. The Archlinux packages, for
example, lacks appropriate handling of the project's license. The wiki
states that licenses should be treated in a certain way, and the IDE
was to complying with that. They are really just paper cuts, but
fixing them outside of the packaging code the IDE uses is way easier.

The debian folder, again, is created on a file by file basis. The IDE
should use dh_make and adapt the initial template of files.That's just
for a start, i only started reviewing this yesterday.

This will *definitely* miss the 3.3 release. We will see how it goes for 3.4

Thanks!

>
> Regards,
>
> --
> Benoît Minisini
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Gambas-devel mailing list
> Gambas-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-devel