(-: Geekerie, logiciel libre et vilain gauchisme

Creating a debian repository

So I have a .deb file, created with CPack, that I want to make available on a public server, so it can be installed on several hosts easily.

It is quite easy to find some documentation on the internet about the various pieces of the process, but a bit more difficult to find the whole process, including the the signature and the verification of the package, that avoid the warning:

WARNING: The following packages cannot be authenticated!
  foo bar
Install these packages without verification [y/N]?

or the use of the --allow-unauthenticated option.

So here is the process:

  • First, create a GPG key, if needed
gpg --gen-key
  • Prepare an empty directory and put the packages in it:
mkdir deb-packages
cp *.deb deb-packages
cd deb-packages
  • Sign the packages
dpkg-sig --sign builder *.deb
  • Export the public key in the directory (the key can also be exported to a key server)
gpg -a --export > great-developer.asc
  • Generate the Packages and Packages.gz files that contains a description of the packages
dpkg-scanpackages . /dev/null | tee Packages | gzip > Packages.gz
  • Generate the Release file that contains the md5 sum of the packages, Packages and Packages.gz files
apt-ftparchive release . > Release
  • Sign the Release file, so we can ensure that everything is done by the expected developer (me)
gpg --yes --armor --output Release.gpg --detach-sig Release
  • Copy all the files on a server where it will be publicly available
cd ..
rsync -av deb-packages/ myserver:/htdocs/deb-packages/

Now on the host where the repository should be used, run

echo "deb http://myserver.com/deb-packages/ ./" >> /etc/apt/sources.list
wget -O - http://myserver.com/deb-packages/great-developer.asc | apt-key add -
apt-get update

Done! The packages can be installed, with a simple apt-get install.

Shrinking VirtualBox vdi files

It is quite easy now to reduce a vdi file, by using the TRIM feature introduced with the SSD. Here is what I’ve done on my virtual machines.

I’ve used this command, with the virtual host down:

VBoxManage storageattach ubuntu --storagectl "SATA" --port 0 --discard on --nonrotational on

then, inside the ubuntu virtual host, I’ve run:

sudo fstrim -v /

Done! This simple operation has freed 5GB!

The VBoxManage command was identical for a Microsoft Windows 7 virtual host, and then I’v run ForceTRIM with administrator privileges from the virtual host - 7 more GB freed on my hard drive!

Of course removing some blocks from the vdi file must have some performance impact. This is not really a problem with linux, because the trim is only done manually (unless the drive is mounted with the discard option). On window it can be disabled with fsutil behavior set DisableDeleteNotify 1, but I don’t know if that’s really worth it. I think I’ll continue to use it with the TRIM enable and see how it works.

Update 2015-01-22

Performances were very bad on windows. Using

fsutil behavior set DisableDeleteNotify 1

helped a lot.

Conclusion: I’ll have to reenable TRIM support and use ForceTRIM from time to time to keep the vdi file as small as possible.

Investigating build failure with both Qt4 and Qt5 installed on Mac OS X

I’m getting these errors while building Qt5 programs on my Mac:

In file included from /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/QApplication:1:
In file included from /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/qapplication.h:48:
In file included from /usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:39:
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qinputmethod.h:82:5: error: unknown type name 'QLocale'
    QLocale locale() const;
    ^
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qinputmethod.h:91:21: error: no type named 'InputMethodQueries' in namespace 'Qt'; did you mean 'InputMethodQuery'?
    void update(Qt::InputMethodQueries queries);
                ~~~~^
/usr/local/include/QtCore/qnamespace.h:1541:10: note: 'InputMethodQuery' declared here
    enum InputMethodQuery {
         ^
In file included from /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/QApplication:1:
In file included from /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/qapplication.h:48:
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:85:12: error: unknown type name 'QWindowList'
    static QWindowList allWindows();
           ^
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:86:12: error: unknown type name 'QWindowList'
    static QWindowList topLevelWindows();
           ^
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:138:12: error: unknown type name 'QFunctionPointer'
    static QFunctionPointer platformFunction(const QByteArray &function);
           ^
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:143:16: error: no type named 'ApplicationState' in namespace 'Qt'
    static Qt::ApplicationState applicationState();
           ~~~~^
/usr/local/opt/qt5/lib/QtGui.framework/Headers/qguiapplication.h:164:38: error: no type named 'ApplicationState' in namespace 'Qt'
    void applicationStateChanged(Qt::ApplicationState state);
                                 ~~~~^

A little search on the web shows this is a problem of conflict with Qt4, and that uninstalling Qt4 should help. Indeed, even just unlinking the qt formula with homebrew makes the build pass:

brew unlink qt

Unfortunately, other programs are now failing:

$ qcachegrind 
dyld: Library not loaded: /usr/local/lib/QtGui.framework/Versions/4/QtGui
  Referenced from: /usr/local/bin/qcachegrind
  Reason: image not found
Trace/BPT trap: 5

So I had to find a better alternative!

clang — and probably all the compilers in their own way — has a flags that shows the headers actually included during the parsing: -H.

In my case, for the Qt5 headers, it shows something like:

.. /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/QApplication
... /usr/local/opt/qt5/lib/QtWidgets.framework/Headers/qapplication.h
.... /usr/local/include/QtCore/qcoreapplication.h
..... /usr/local/include/QtCore/qobject.h

The command line looks like that (I’ve suppressed everything not needed to make it a bit easier to read):

c++ -isystem /usr/local/opt/qt5/lib/QtCore.framework/Headers \
  -iframework /usr/local/opt/qt5/lib \
  -isystem /usr/local/opt/qt5/lib/QtWidgets.framework/Headers \
  -isystem /usr/local/include \ 
  ...

The first two lines of includes are ok, but then, at the third line, it switched to Qt4 includes (note the lack of qt5 in the path).

In qapplication.h, the header QtCore/qcoreapplication.h is included. According to Apple’s documentation on frameworks, this means that the qcoreapplication.h header could be located in the Qt5 framework QtCore.framework/Headers (or QtCore/PrivateHeaders), if QtCore.framework is in /System/Library/Frameworks or /Library/Frameworks, or if the base directory of QtCore.framework is passed to the compiler with the flag -framework or -iframework.

QtCore.framework is in /usr/local/opt/qt5/lib, which is not one of the default framework directories, but the flag -iframework is properly used:

-iframework /usr/local/opt/qt5/lib

so this is why it works when there is Qt5 alone.

However, when Qt4 is also installed, the compiler seems to first look at the normal includes before looking at the frameworks, and so the first QtCore/qcoreapplication.h found is the one from Qt4, which is in the include dir for other libraries: /usr/local/include.

Qt5 also has its own include directory that contain the QtCore directory (and others) as Qt4 does, and including that directory before the Qt4 directory is enough to fix the build.

With CMake unfortunately, it is not included when using the standard find_package(Qt5...) commands. It has to be done by hand, with

include_directories(BEFORE /usr/local/opt/qt5/include)

or if you are not the developer, you can set the CMAKE_CXX_FLAGS option to -I/usr/local/opt/qt5/include:

cmake -DCMAKE_CXX_FLAGS:STRING=-I/usr/local/opt/qt5/include ..

Removing bzip2 dependency in boost

I had this error during boost build yesterday:

gcc.compile.c++ bin.v2/libs/iostreams/build/gcc-4.8/release/link-static/threading-multi/bzip2.o
libs/iostreams/src/bzip2.cpp:20:56: fatal error: bzlib.h: No such file or directory
 #include "bzlib.h"  // Julian Seward's "bzip.h" header.
                                                        ^
compilation terminated.

I don’t need any bzip2 related feature for my project, and the iostreams documentation says I can disable the bzip2 support with the NO_BZIP2 Boost.build variable, but it doesn’t give any example of how to use it.

So here is how to use it: with the -s flag. For example:

./b2 -s NO_BZIP2=1

Utiliser un VPN avec transmission, FranceTV Pluzz et Arte+7

De nombreux vendeurs de VPN fournissent des solutions efficaces contre la surveillance systématique façon Hadopi, en changeant l’origine apparente de l’accès internet. Un accès hors de France n’intéresse plus Hadopi.

Malheureusement, si il est simple de configurer son linux pour utiliser un VPN, il est moins simple de faire cohabiter différents outils sur la même machine.

Les services de replay de télévision par exemple, utilisent l’origine géographique de la connexion internet pour autoriser ou refuser l’accès. Si la connexion parait venir hors de France, l’accès sera impossible à FranceTV pluzz. Arte étant une chaîne franco-allemande, Arte+7 autorise aussi l’accès depuis l’Allemagne.

Pour pouvoir accéder à FranceTV pluzz et Arte+7, le réseau est configuré de façon à faire passer tout le trafic vers l’extérieur par le VPN jusqu’en Allemagne, à l’exception des accès aux services de replay FranceTV pluzz. De cette façon, par défaut, toutes les communications seront anonymisées par le VPN, y compris tout ce à quoi on a pas pensé au moment de la configuration du VPN. Seules certaines adresses correspondant à FranceTV pluzz peuvent passer par la connexion hors VPN.

Et pour éviter les oublis, transmission est démarré et arrêté en même temps que le VPN.

Configurer l’accès par VPN

Les exemples listés ici utilisent hidemyass et OpenVPN. Plusieurs serveurs sont disponibles en Allemagne, dans plusieurs villes. Les pings varient sensiblement entre les différentes villes disponibles. Pour moi, celle donnant le ping le plus faible est Francfort.

Télécharger la configuration de base

On télécharge d’abord un fichier de configuration correspondant à la ville désirée. Par exemple, https://www.hidemyass.com/vpn-config/UDP/Germany.Hesse.Frankfurt_LOC1S1.UDP.ovpn, et on le sauve dans /etc/openvpn/hma.conf.

Utiliser plusieurs serveurs

Pour utiliser tous les serveurs disponibles dans cette ville, on ajoute des commandes remote correspondant à chaque serveur et la commande remote-random qui permet de choisir un des serveur au hasard. De cette façon, OpenVPN essaiera différents serveurs en cas de problème de connexion. Pour Francfort, on obtient :

remote-random
remote 62.113.202.130 53
remote 62.113.251.2 53
remote 62.113.206.2 53
remote 62.113.206.130 53
remote 62.113.254.130 53
remote 62.113.251.130 53
remote 212.83.40.2 53
remote 212.83.59.130 53

Authentification automatique

Pour que l’authentification se fasse sans demander de mot de passe, on ajoute un fichier /etc/openvpn/hmauser.pass contenant le login et le mot de passe séparé par un retour à la ligne :

monLogin
monMotDePasse

Il vaut mieux s’assurer que le fichier n’est accessible que par l’utilisateur root :

chown root /etc/openvpn/hmauser.pass
chmod 600 /etc/openvpn/hmauser.pass

Ensuite, dans le fichier /etc/openvpn/hma.conf, on remplace auth-user-pass par auth-user-pass /etc/openvpn/hmauser.pass

Démarrage de la connexion VPN

Cette étape dépend de la distribution linux utilisée. Sur Ubuntu, c’est

sudo service openvpn start

À ce stade, l’ensemble des connexions passent par le VPN, ce qu’on peut vérifier avec la commande traceroute ou en accédant à un service permettant de localiser l’adresse IP comme celui-ci. Avec la configuration précédente, la connexion doit apparaitre comme venant d’Allemagne.

À ce stade, il est normalement impossible d’accéder au contenu de FranceTV pluzz puisque la connexion semble venir d’Allemagne.

Accéder à FranceTV pluzz

Pour accéder à FranceTV pluzz, on force certaines adresses IP à ne pas passer par le VPN. Pour ça on crée deux nouveaux fichiers :

  • /etc/openvpn/hma-up.sh :
#!/bin/bash

for ip in 77.67.21.223 77.67.11.114 5.178.43.9 ; do
  /sbin/ip route add $ip/32 via 192.168.1.1
done
  • /etc/openvpn/hma-down.sh :
#!/bin/bash

for ip in 77.67.21.223 77.67.11.114 5.178.43.9 ; do
  /sbin/ip route del $ip/32 via 192.168.1.1
done

Il faut s’assurer que ces fichiers sont exécutables :

chmod a+x /etc/openvpn/hma-*.sh

Enfin on ajoute ces lignes au fichier /etc/openvpn/hma.conf :

script-security 2
up /etc/openvpn/transmission-up.sh
down /etc/openvpn/transmission-down.sh

enfin on ajoute ces lignes à /etc/hosts, pour forcer la résolution des noms utilisée par FranceTV pluzz à une seule adresse IP :

77.67.21.223    webservices.francetelevisions.fr
77.67.11.114    medias2.francetv.fr
5.178.43.9      ftvodhdsecz-f.akamaihd.net

À ce stade, FranceTV pluzz doit à nouveau être accessible.

Démarrer transmission en même temps que la connexion VPN

Pour démarrer transmission en même temps que le VPN, on ajoute tout simplement

/etc/init.d/transmission-daemon start

dans /etc/openvpn/hma-up.sh, et

/etc/init.d/transmission-daemon stop

dans /etc/openvpn/hma-down.sh.

Il faut également s’assurer que le service qui démarre transmission est désactivé. Toujours sur ubuntu :

sudo update-rc.d transmission-daemon remove