HOWTO Keep a network of Debians up-to-date

This article was first written in August 2005 for the BeezNest technical
website (http://glasnost.beeznest.org/articles/289).

Due to the unrivaled apt tools, Debian is probably the easiest GNU/Linux distribution to keep up-to-date, at least when you have a fast Internet connection.

In a network of many machines, the goal is slightly different though: you probably want to take as little bandwidth as possible away from users.

cron-apt

The first part of the solution, and the easiest to setup, is to install package cron-apt which will download the packages early in the morning from cron, when users are not at work. No need to configure anything, it works out of the box to do the equivalent of an (apt-get) update and download the packages, without installing anything for you. You keep control. You can configure it to make it install some or all updates if you want.

apt-proxy

The second part is to setup a proxy specialized in getting Debian packages for you, only once for all machines. This proxy is apt-proxy. The second machine requesting the same package will take only a fraction of the time needed to download it from the Internet. It will also spare your Internet connection quota, so even for a few Debian machines, it may prove useful. You only need some spare disk space on a Debian machine on your network to use it (it is also performance-hungry when working). It is slightly trickier to setup, because you need to carefully define your sources. Here is a proposed configuration for it:

[DEFAULT]
;; All times are in seconds, but you can add a suffix
;; for minutes(m), hours(h) or days(d)

;; Server IP to listen on
;address = 192.168.0.254

;; Server port to listen on
port = 9999

;; Control files (Packages/Sources/Contents) refresh rate
;;
;; Minimum time between attempts to refresh a file
min_refresh_delay = 1h

;; Minimum age of a file before attempting an update (NOT YET IMPLEMENTED)
;min_age = 23h

;; Uncomment to make apt-proxy continue downloading even if all
;; clients disconnect.  This is probably not a good idea on a
;; dial up line.
;; complete_clientless_downloads = 1
complete_clientless_downloads = 1

;; Debugging settings.
;; for all debug information use this:
;; debug = all:9
debug = all:4 db:0

;; Debugging remote python console
;; Do not enable in an untrusted environment
;telnet_port = 9998
;telnet_user = apt-proxy
;telnet_password = secret

;; Network timeout when retrieving from backend servers
timeout = 15

;; Cache directory for apt-proxy
cache_dir = /var/cache/apt-proxy

;; Use passive FTP? (default=on)
;passive_ftp = on

;; Use HTTP proxy?
;http_proxy = host:port

;; Enable HTTP pipelining within apt-proxy (for test purposes)
;disable_pipelining=0

;;--------------------------------------------------------------
;; Cache housekeeping

;; Time to perform periodic housekeeping:
;;  - delete files that have not been accessed in max_age
;;  - scan cache directories and update internal tables
cleanup_freq = 1d

;; Maximum age of files before deletion from the cache (seconds)
max_age = 120d

;; Maximum number of versions of a .deb to keep per distribution
max_versions = 3

;; Add HTTP backends dynamicaly if not already defined? (default=on)
;dynamic_backends = on

;;---------------------------------------------------------------
;;---------------------------------------------------------------
;; Backend servers
;;
;; Place each server in its own [section]

[debian]
;; The main Debian archive
;; You can override the default timeout like this:
;timeout = 30

;; Rsync server used to rsync the Packages file (NOT YET IMPLEMENTED)
;;rsyncpackages = rsync://ftp.de.debian.org/debian

;; Backend servers, in order of preference
backends =
        ftp://ftp.skynet.be/debian
        ftp://ftp.belnet.be/debian

http://ftp.de.debian.org/debian


http://ftp2.de.debian.org/debian

        ftp://ftp.uk.debian.org/debian

[debian-non-US]
;; Debian debian-non-US archive
;timeout will be the global value
backends =
        ftp://ftp.skynet.be/debian-non-US
        ftp://ftp.belnet.be/debian-non-US

http://ftp.uk.debian.org/debian-non-US


http://ftp.de.debian.org/debian-non-US

        ftp://ftp.uk.debian.org/debian

[security]
;; Debian security archive
backends =
        ftp://ftp.skynet.be/debian-security
        ftp://ftp.belnet.be/debian-security

http://security.debian.org/debian-security


http://ftp2.de.debian.org/debian-security

;[apt-proxy]
;; Apt-proxy new versions
;backends = http://apt-proxy.sourceforge.net/apt-proxy

;[backports.org]
;; backports.org
;backends = http://backports.org/debian

[blackdown]
;; Blackdown Java
backends =
;       ftp://ftp.easynet.be/blackdown/debian
;       http://ftp.gwdg.de/pub/languages/java/linux/debian
        ftp://ftp.tux.org/java/debian/

;[debian-people]
;; people.debian.org
;backends = http://people.debian.org

;[emdebian]
;; The Emdebian project
;backends = http://emdebian.sourceforge.net/emdebian

;[rsync]
;; An example using an rsync server.  This is not recommended
;; unless http is not available, because rsync is only more
;; efficient for transferring uncompressed files and puts much
;; more overhead on the server.  See the rsyncpackages parameter
;; for a way of rsyncing just the Packages files.
;backends = rsync://ftp.uk.debian.org/debian

[java]
backends =
        ftp://ftp2.skynet.be/pub/ftp.blackdown.org/debian

[multimedia]
backends =
        ftp://ftp.nerim.net/debian-marillat

[bxlug]
backends =

http://apt.bxlug.be/

An example matching sources.list file for Sarge would be:

# See sources.list(5) for more information, especialy
# Remember that you can only use http, ftp or file URIs
# CDROMs are managed through the apt-cdrom tool.

# Standard Debian sources (with contrib and non-free)
deb http://apt-proxy:9999/debian/ sarge main contrib non-free
deb-src http://apt-proxy:9999/debian/ sarge main contrib non-free

# Debian Security updates
deb http://apt-proxy:9999/security/ sarge/updates main contrib non-free

# BxLUG sources
deb http://apt.bxlug.be/ experimental-sarge/
deb http://apt.bxlug.be/sarge/ main/
deb http://apt.bxlug.be/sarge/ contrib/
deb http://apt.bxlug.be/sarge/ non-free/
deb http://apt.bxlug.be/sarge/ not-debian/

# Multimedia stuff from Marillat
deb http://apt-proxy:9999/multimedia/ sarge main

# Java gcc-3.2
deb http://apt-proxy:9999/java/ sarge non-free

apticron

To be kept informed (by e-mail) of available updates, just install the package of that name.

Special note about web proxies

Using a general-purpose web proxy is generally a bad idea, because they are not meant to store large files. If you want to do it anyway, you just need to define the matching environment variable amongst these: HTTP_PROXY, HTTPS_PROXY, FTP_PROXY.

ntop

This article was first written in August 2005 for the BeezNest technical
website (http://glasnost.beeznest.org/articles/287).

ntop

ntop is a nice tool with a nice web interface (and a CLI interface also).

Sadly, it is currently leaking too much memory and CPU cycles to let it run for a long time on a server.

Networked printing infrastructure using CUPS on Debian

This article was first written in August 2005 for the BeezNest technical
website (http://glasnost.beeznest.org/articles/286).

In a networked infrastructure, with many workstations needing to print (or generate PDF or PS files, or send faxes), CUPS may help a lot, especially with its automatic network-browsing features.

Let’s see how to set that up.

CUPS Server

First, you probably want to setup a server to share printers. Those printers may be networked themselves (ie embed a print server or be connected to an external one) or local (parallel, USB or serial) to the server. The print server types that CUPS may use to access its printers are:

• lpd/lpr
• IPP
• SMB/CiFS
• AppSocket/HP JetDirect

You only need to install CUPS server and printers’ definitions in the form of PPD files (which may in turn require some tools to work correctly). Configure the CUPS server to allow browsing in both directions (be careful how you do it though, as it may represent a security risk).

CUPS clients

The CUPS clients, if they don’t need to share printers themselves, do not need to host a full-blown CUPS server, but only use libraries and/or client utilities (like lpr and/or lp). To tell the CUPS library to use a remote CUPS server, it suffice to fulfill the CUPS_SERVER environment variable of the machine or the user’s profile.

Light print server

If you only need to give access to a local printer on a workstation, maybe a better idea would be to avoid running a full CUPS server on it, and run a light print server simulating an HP JetDirect server. You would avoid at the same time having to install all it’s dependencies, which are numerous and may eat a lot of disk space. Hopefully, p910nd exists just for that. It takes almost no memory and is usable from inetd (to load it into memory only when needed) or as daemon. You would lose the auto-detection, but it is not a huge work to create once the printer on the central CUPS server.

sort

This article was first written in August 2005 for the BeezNest technical
website (http://glasnost.beeznest.org/articles/285).

Sorting a passwd database on UNIX based on user ids:

sort -t: -k 3 -n /etc/passwd