Ce site est optimisé pour être consulté depuis un navigateur moderne dans lequel JavaScript est activé.

Migration de Ubuntu vers Debian

nicolas

https://github.com/club-1/hosting/issues/57

Ça ne sera pas de tout repos.

À faire en prévision :

Sauvegarde de la liste des paquets installés.
- Une solution basée sur apt-mark showmanual pourrait faire l’affaire.
Suppression autant que possible de dépôts externes.
Suppression autant que possible de paquets Ubuntu Only (presque fini).

nicolas

TODO list

Étapes préparatoires

Environnement de prod

Export de la liste des paquets avec apt-mark showmanual > installed-packages.txt
Filtrages automatiques et manuels de la liste pour ne garder que ceux que l’on veut réellement

Préparation du merge de `/etc` dans l’environnement de test

Merge des dossiers de config /etc (après l’installation et le configuration des paquets.
- cd /etc
- sudo su
- git remote add ubuntu /home/nicolas/etc.git
- git remote update
- git merge --allow-unrelated-histories ubuntu/master
- git mergtool (résoudre tous les confilts)
  - les fichiers subgid et subuid sont étranges. Sont-ils vraiment utilisés ? Conservation de la version d’ubuntu pour le moment. → Supprimés
  - remplacer plus tard le symlink systemd enable coturn
- git clean -i (suppression des fichiers *.org créés par nvimdiff)
- git reset (unstage tout)
- git rev-parse ubuntu/master > .git/MERGE_HEAD (restauration de l’info de merge après le reset)
- git add -u . (ajout des fichiers modifiés uniquement)
- git add -N . (indique à git qu’on compte ajouter le reste des fichiers à l’index, pour pouvoir add en interactive)
- git add -p . (add interactif)
  - peut-être installer à nouveau smartmontools ?
  - voir sshd_config.merge-error
  - remplacer php7.4-fpm.service par un override systemd
  - ajout de pm2-nicolas.service à nouveau → pm2 ne sera pas réinstallé, à la place une unit systemd user pourra être faite.
  - installer à nouveau prometheus-node-exporter-collectors → pas nécessaire pour le moment
- git merge --continue (finalisation du merge)
- cp -r /home/nicolas/debian/.git .
- git reset --hard
- etckeeper init

souvenirs de cette étape

nicolas@X260-NICOLAS:~/Projets/Club1/debian> [master ≡]
$ git merge ubuntu/master --allow-unrelated-histories | grep CONFLICT | wc -l
581
nicolas@X260-NICOLAS:~/Projets/Club1/debian> [master ≡|MERGING +1478 ~3 -0 | +581 ~0 -0 !]
$ git mergetool

nicolas@X260-NICOLAS:~/Projets/Club1/debian> [master ≡|MERGING +275 ~82 -0 | +622 ~0 -0 !]
$ git merge --continue 
[master 1b2beba] Merge de la config de ubuntu vers debian
nicolas@X260-NICOLAS:~/Projets/Club1/debian> [master ↑824 +622 ~0 -0 !]
$ git log --graph
*   commit 1b2bebaa0b61813cd4cd5c93df38a5357291eeca (HEAD -> master)
|\  Merge: 2cc4727 aee9ff2
| | Author: Nicolas Peugnet <nicolas@club1.fr>
| | Date:   Sun Jan 22 18:25:23 2023 +0100
| | 
| |     Merge de la config de ubuntu vers debian
| |     
| |     En ne gardant que les fichiers véritablement modifiés pour le serveur
| |     CLUB1.
| | 
| * commit aee9ff2d5bf2d1aa73ab371d81b327afe6d4b58e (ubuntu/master)
| | Author: n-peugnet <n.peugnet@free.fr>
| | Date:   Sat Jan 14 08:47:59 2023 +0100
| | 
| |     apt: update grafana key and remove old trusted keys
| | 
...skipping...
| | 
| * commit 31a6a1cacaee510004caf8c4d6d5641f52845a57
| | Author: n-peugnet <n.peugnet@free.fr>
| | Date:   Wed May 20 21:31:22 2020 +0200
| | 
| |     committing changes in /etc after apt run
| |     
| |     Package changes:
| |     +freeipmi 1.4.11-1.1ubuntu4.1 amd64
| |     +freeipmi-bmc-watchdog 1.4.11-1.1ubuntu4.1 amd64
| |     +freeipmi-ipmidetect 1.4.11-1.1ubuntu4.1 amd64
| |     +freeipmi-tools 1.4.11-1.1ubuntu4.1 amd64
| |     +libipmiconsole2 1.4.11-1.1ubuntu4.1 amd64
| |     +libipmidetect0 1.4.11-1.1ubuntu4.1 amd64
| | 
| * commit ae350e3367395c770fc64f88b827c7bf94c8b5e2
|   Author: n-peugnet <n.peugnet@free.fr>
|   Date:   Wed May 20 17:42:01 2020 +0200
|   
|       Initial commit
| 
* commit 2cc47276ebf7f25b00e910803430482a362b4c4f (origin/master, origin/HEAD)
| Author: root <root@club1.club1.fr>
| Date:   Sat Jan 21 20:06:01 2023 +0100
| 
|     dpkg configuration finalized
| 
* commit 53508870798209205ec193e891a509f93e696467
  Author: nicolas <nicolas@club1.club1.fr>
  Date:   Sat Jan 21 19:52:09 2023 +0100
  
      Initial commit

Environnement de test (étapes à réaliser en production)

Avant tout, définir un mot de passe root sur club1, redémarrer et lancer une sauvegarde (borgmatic dans screen) ~= 2h15

Phase 1 (installation du système)

Language: English
Location: Europe/France
Locale: en-US
Hostname: club1.fr
Mot de passe root: aucun
Utilisateur: Nicolas
Partitionning: Guided (All files in one partition), puis rétrecissement de la partition racine pour mettre 4Gio de Swap
Mirrors: deb.debian.org
Popcon: yes
Software to install: « SSH server », « Standard system utilities »

Phase 2 EN TANT QUE ROOT (restauration de certains fichiers du système précédent)

dpkg-reconfigure locales selection des locales en-US, fr-FR et en-GB en UTF-8. Principalement pour ne pas avoir de messages d’erreurs lors des étapes suivantes.
Mount de l’ISO d’install
apt install vim borgbackup screen
mkdir ~/.ssh
echo 'nas.jeanpierre.moe ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdzQyGgLl9YgPVGeyCxRnOdswraNagx6xSFROaxjTjrM2WwXlFhxLXOxounGKUYBA2l5McCgNh8cneJjKOunmQ=' >> ~/.ssh/known_hosts
borg list club1@nas.jeanpierre.moe:backups/system
cd / && borg extract club1@nas.jeanpierre.moe:backups/system::<archive> etc/shadow etc/passwd etc/group etc/gshadow root/apt-packages.txt etc/borgmatic.d/system.yaml root/.ssh

Phase 3 EN TANT QUE ROOT (installation des paquets)

mkdir /etc/systemd/system-preset && echo 'disable *' > /etc/systemd/system-preset/disable-all.preset
apt install extrepo
extrepo enable grafana
extrepo enable jellyfin
extrepo enable matrix
apt update
dans screen: cat /root/apt-packages.txt | DEBIAN_FRONTEND=noninteractive xargs apt install --no-install-recommends -y

Phase 4 EN TANT QUE ROOT (restauration totale)

cp /etc/fstab /root (sauvegarde du nouveau fichier fstab)
dans screen: borgmatic extract --archive latest --repository=club1@nas.jeanpierre.moe:backups/system --destination / ~= 40min
Récupération du etc mergé, cette étape ne sera pas nécessaire pour les futures restaurations :
- cd /etc && git add . && git reset --hard debian/master
- etckeeper init
Restauration de certains fichiers spéciaux :
- cp /root/fstab /etc
- vérifier que le fstab est cohérent et l’ajouter au git au passage avec git add -p + git commit
- mkdir /var/spool/postfix/opendkim && chown opendkim:postfix /var/spool/postfix/opendkim
- ln -s /usr/local/src/hydrian/TTRSS-Auth-LDAP/plugins/auth_ldap /usr/share/tt-rss/www/plugins/
Restauration de la bdd ldap :
- systemctl stop slapd
- slapadd -l /root/ldap_data.ldif
- chown -R openldap:openldap /var/lib/ldap
- systemctl restart slapd nslcd
Restauration de la bdd des alias email :
- newaliases
Reload de la config de systemd :
- systemctl daemon-reload
Restauration des bdd mariadb :
- systemctl start mariadb
- dans screen: borgmatic restore --archive latest --repository=club1@nas.jeanpierre.moe:backups/mariadb ~= 30mins
- mariadb < /root/mariadb-users.sql
Restauration des bdd postgres :
- systemctl start postgresql
- sudo -u postgres psql -c 'create role root with superuser login;'
- sudo -u postgres psql -c 'create database root with owner root;'
- dans screen: borgmatic restore --archive latest --repository=club1@nas.jeanpierre.moe:backups/postgres ~= 6h
Pour que le mount de home fonctionne :
- sudo rm -rf /home/*
Redémarrage

Notes

Exceptions lors de l’installation des paquets

remplacement et suppression de certains paquets :

E: Unable to locate package apport-symptoms --> X
E: Package 'busybox-initramfs' has no installation candidate --> X
           'busybox-static' --> X
E: Package 'libmariadbclient18' has no installation candidate --> X
           'libmariadbd19' --> X
E: Unable to locate package libmpdec2 --> X
E: Package 'libmysqlclient-dev' has no installation candidate --> X
E: Package 'libreadline-gplv2-dev' has no installation candidate --> X
E: Package 'libreadline5' has no installation candidate --> X
E: Unable to locate package linux-generic --> X
                            linux-base --> X
E: Unable to locate package linux-image-5.4.0-121-generic --> X
E: Couldn't find any package by glob 'linux-image-5.4.0-121-generic'
E: Unable to locate package overlayroot --> X
E: Unable to locate package plymouth-theme-ubuntu-text --> X
E: Package 'run-one' has no installation candidate
E: Package 'ubuntu-keyring' has no installation candidate --> X
E: Unable to locate package unison-all --> X

notes

Note, selecting 'libcom-err2' instead of 'libcomerr2'
Note, selecting 'libgcc-s1' instead of 'libgcc1'

paquets déjà présents

libacl1 is already the newest version (2.2.53-10).
adduser is already the newest version (3.118).
apparmor is already the newest version (2.13.6-10).
apparmor set to manually installed.
libapparmor1 is already the newest version (2.13.6-10).
apt is already the newest version (2.2.4).
apt-transport-https is already the newest version (2.2.4).
apt-utils is already the newest version (2.2.4).
apt-listchanges is already the newest version (3.24).
libattr1 is already the newest version (1:2.4.48-6).
libaudit-common is already the newest version (1:3.0-2).
libaudit1 is already the newest version (1:3.0-2).
base-files is already the newest version (11.1+deb11u6).
base-passwd is already the newest version (3.5.51).
bash is already the newest version (5.1-2+deb11u1).
bash-completion is already the newest version (1:2.11-2).
bind9-dnsutils is already the newest version (1:9.16.33-1~deb11u1).
bind9-host is already the newest version (1:9.16.33-1~deb11u1).
bzip2 is already the newest version (1.0.8-4).
libbz2-1.0 is already the newest version (1.0.8-4).
ca-certificates is already the newest version (20210119).
libdebconfclient0 is already the newest version (0.260).
console-setup is already the newest version (1.205).
console-setup-linux is already the newest version (1.205).
console-setup-linux set to manually installed.
keyboard-configuration is already the newest version (1.205).
coreutils is already the newest version (8.32-4+b1).
cpio is already the newest version (2.13+dfsg-4).
cron is already the newest version (3.0pl1-137).
libsasl2-modules is already the newest version (2.1.27+dfsg-2.1+deb11u1).
libsasl2-modules set to manually installed.
dash is already the newest version (0.5.11+git20200708+dd9ef66-5).
libdb5.3 is already the newest version (5.3.28+dfsg1-0.8).
dbus is already the newest version (1.12.24-0+deb11u1).
debconf is already the newest version (1.5.77).
debconf-i18n is already the newest version (1.5.77).
debianutils is already the newest version (4.11.2).
diffutils is already the newest version (1:3.7-5).
distro-info-data is already the newest version (0.51+deb11u3).
distro-info-data set to manually installed.
dmidecode is already the newest version (3.3-2).
dpkg is already the newest version (1.20.12).
e2fsprogs is already the newest version (1.46.2-2).
libcom-err2 is already the newest version (1.46.2-2).
libss2 is already the newest version (1.46.2-2).
libexpat1 is already the newest version (2.2.10-2+deb11u5).
libexpat1 set to manually installed.
file is already the newest version (1:5.39-3).
libmagic1 is already the newest version (1:5.39-3).
libmagic1 set to manually installed.
findutils is already the newest version (4.8.0-1).
libgcc-s1 is already the newest version (10.2.1-6).
libstdc++6 is already the newest version (10.2.1-6).
gettext-base is already the newest version (0.21-4).
libc-bin is already the newest version (2.31-13+deb11u5).
libc6 is already the newest version (2.31-13+deb11u5).
locales is already the newest version (2.31-13+deb11u5).
libgmp10 is already the newest version (2:6.2.1+dfsg-1+deb11u1).
gpgv is already the newest version (2.2.27-2+deb11u2).
libgnutls30 is already the newest version (3.7.1-5+deb11u2).
libgpm2 is already the newest version (1.20.7-8).
libgpm2 set to manually installed.
grep is already the newest version (3.6-1).
groff-base is already the newest version (1.22.4-6).
grub-common is already the newest version (2.06-3~deb11u5).
gzip is already the newest version (1.10-4+deb11u1).
hostname is already the newest version (3.23).
ifupdown is already the newest version (0.8.36).
init is already the newest version (1.60).
init-system-helpers is already the newest version (1.60).
initramfs-tools is already the newest version (0.140).
installation-report is already the newest version (2.78).
iproute2 is already the newest version (5.10.0-4).
iputils-ping is already the newest version (3:20210202-1).
isc-dhcp-client is already the newest version (4.4.1-2.3+deb11u1).
isc-dhcp-common is already the newest version (4.4.1-2.3+deb11u1).
kbd is already the newest version (2.3.0-3).
kbd set to manually installed.
libkeyutils1 is already the newest version (1.6.1-2).
klibc-utils is already the newest version (2.0.8-6.1).
klibc-utils set to manually installed.
libklibc is already the newest version (2.0.8-6.1).
libklibc set to manually installed.
kmod is already the newest version (28-1).
libkmod2 is already the newest version (28-1).
krb5-locales is already the newest version (1.18.3-6+deb11u3).
less is already the newest version (551-2).
libbsd0 is already the newest version (0.11.3-1).
libcap2 is already the newest version (1:2.44-1).
libcap2-bin is already the newest version (1:2.44-1).
libestr0 is already the newest version (0.1.10-2.1+b1).
libgcrypt20 is already the newest version (1.8.7-6).
libgpg-error0 is already the newest version (1.38-2).
liblocale-gettext-perl is already the newest version (1.07-4+b1).
libmnl0 is already the newest version (1.0.4-3).
libseccomp2 is already the newest version (2.5.1-1+deb11u1).
libselinux1 is already the newest version (3.1-3).
libsemanage-common is already the newest version (3.1-1).
libsemanage1 is already the newest version (3.1-1+b2).
libsepol1 is already the newest version (3.1-1).
libtasn1-6 is already the newest version (4.16.0-2+deb11u1).
libtext-charwidth-perl is already the newest version (0.04-10+b1).
libtext-iconv-perl is already the newest version (1.7-7+b1).
libtext-wrapi18n-perl is already the newest version (0.06-9).
libusb-0.1-4 is already the newest version (2:0.1.12-32).
libusb-0.1-4 set to manually installed.
logrotate is already the newest version (3.18.0-2+deb11u1).
lsb-base is already the newest version (11.1.0).
lsb-release is already the newest version (11.1.0).
lsb-release set to manually installed.
lsof is already the newest version (4.93.2+dfsg-1.1).
dmsetup is already the newest version (2:1.02.175-2.1).
libdevmapper1.02.1 is already the newest version (2:1.02.175-2.1).
liblz4-1 is already the newest version (1.9.3-2).
man-db is already the newest version (2.9.4-2).
manpages is already the newest version (5.10-1).
mawk is already the newest version (1.3.4.20200120-2).
mime-support is already the newest version (3.66).
nano is already the newest version (5.4-2+deb11u2).
ncurses-base is already the newest version (6.2+20201114-2).
ncurses-bin is already the newest version (6.2+20201114-2).
ncurses-term is already the newest version (6.2+20201114-2).
netbase is already the newest version (6.3).
telnet is already the newest version (0.17-42).
libnewt0.52 is already the newest version (0.52.21-4+b3).
whiptail is already the newest version (0.52.21-4+b3).
nftables is already the newest version (0.9.8-3.1+deb11u1).
openssh-client is already the newest version (1:8.4p1-5+deb11u1).
openssh-server is already the newest version (1:8.4p1-5+deb11u1).
openssh-server set to manually installed.
openssh-sftp-server is already the newest version (1:8.4p1-5+deb11u1).
openssh-sftp-server set to manually installed.
openssl is already the newest version (1.1.1n-0+deb11u3).
openssl set to manually installed.
libp11-kit0 is already the newest version (0.23.22-1).
libpam-modules is already the newest version (1.4.0-9+deb11u1).
libpam-modules-bin is already the newest version (1.4.0-9+deb11u1).
libpam-runtime is already the newest version (1.4.0-9+deb11u1).
libpam0g is already the newest version (1.4.0-9+deb11u1).
pciutils is already the newest version (1:3.7.0-5).
libpcre2-8-0 is already the newest version (10.36-2+deb11u1).
perl is already the newest version (5.32.1-4+deb11u2).
perl-base is already the newest version (5.32.1-4+deb11u2).
libpopt0 is already the newest version (1.18-2).
popularity-contest is already the newest version (1.71).
libpq5 is already the newest version (13.9-0+deb11u1).
libpq5 set to manually installed.
procps is already the newest version (2:3.3.17-5).
python3 is already the newest version (3.9.2-3).
python3 set to manually installed.
readline-common is already the newest version (8.1-1).
rsyslog is already the newest version (8.2102.0-2+deb11u1).
sed is already the newest version (4.7-1).
sensible-utils is already the newest version (0.0.14).
login is already the newest version (1:4.8.1-1).
passwd is already the newest version (1:4.8.1-1).
libslang2 is already the newest version (2.3.2-5).
sudo is already the newest version (1.9.5p2-3).
libsystemd0 is already the newest version (247.3-7+deb11u1).
libudev1 is already the newest version (247.3-7+deb11u1).
systemd is already the newest version (247.3-7+deb11u1).
systemd-sysv is already the newest version (247.3-7+deb11u1).
udev is already the newest version (247.3-7+deb11u1).
sysvinit-utils is already the newest version (2.96-7+deb11u1).
tar is already the newest version (1.34+dfsg-1).
tasksel is already the newest version (3.68+deb11u1).
libwrap0 is already the newest version (7.6.q-31).
libwrap0 set to manually installed.
traceroute is already the newest version (1:2.1.0-2+b1).
tzdata is already the newest version (2021a-1+deb11u8).
ucf is already the newest version (3.0043).
usbutils is already the newest version (1:013-3).
bsdutils is already the newest version (1:2.36.1-8+deb11u1).
libblkid1 is already the newest version (2.36.1-8+deb11u1).
libmount1 is already the newest version (2.36.1-8+deb11u1).
libsmartcols1 is already the newest version (2.36.1-8+deb11u1).
libuuid1 is already the newest version (2.36.1-8+deb11u1).
mount is already the newest version (2.36.1-8+deb11u1).
util-linux is already the newest version (2.36.1-8+deb11u1).
vim is already the newest version (2:8.2.2434-3+deb11u1).
vim-common is already the newest version (2:8.2.2434-3+deb11u1).
vim-runtime is already the newest version (2:8.2.2434-3+deb11u1).
vim-runtime set to manually installed.
vim-tiny is already the newest version (2:8.2.2434-3+deb11u1).
wget is already the newest version (1.21-1+deb11u1).
xkb-data is already the newest version (2.29-2).
xkb-data set to manually installed.
liblzma5 is already the newest version (5.2.5-2.1~deb11u1).
xz-utils is already the newest version (5.2.5-2.1~deb11u1).
zlib1g is already the newest version (1:1.2.11.dfsg-2+deb11u2).

nicolas

Problèmes lors de la migration :

slapd était tout de même démarré malgré le preset disabled, il faut le stopper avant de faire le slapadd.
nslcd necessitait un restart.
la bdd mysql de MariaDB n’était pas sauvegardé, il a fallu recréer les users et leurs permissions à la main
icecast2: necessaire de relancer dpkg-reconfigure icecast2 et de dire oui à la configuration (probablement plus necessaire avec les dernières sauvegardes)
tt-rss-auth-ldap nécessite de créer un lien symbolique dans /usr/share/tt-rss/www/plugins/
mail.club1.fr a un problème de permissions, le dossier public_html a été ajouté dans une version plus récente du paquet debian, il fallait mettre à jour le vhost
initramfs-tools/conf.d/resume doit être copié au même titre que fstab juste après l’install, car il conntient un UUID de partition, ou alors complètement désactiver l’hibernation (beaucoup plus simple)

nicolas

Cette fois c’est fini. Un petit article de récap devrait arriver bientôt ! (Mais clairement pas tout de suite)

vincent

Wwoouhhhaaa ! Toutes ces cases cochées !! C’est superbe ! bravo @nicolas et @etienne !!

🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

nicolas

Le processus de restauration est maintenant rangé ici : https://club1.fr/docs/fr/outils/sauvegardes.html#processus-de-restauration