Debian
liblicense 0.8.1: The bugfixiest release ever
asheesh, December 25th, 2008
I’m greatly pleased to announce liblicense 0.8.1. Steren and Greg found a number of major issues (Greg found a consistent crasher on amd64, and Steren found a consistent crasher in the Python bindings). These issues, among
some others, are fixed by the wondrous liblicense 0.8.1. I mentioned to Nathan Y. that liblicense is officially “no longer ghetto.”
The best way enjoy liblicense is from our Ubuntu and Debian package repository, at http://mirrors.creativecommons.org/packages/. More information on what liblicense does is available on our wiki page about liblicense. You can also get them in fresh Fedora 11 packages. And the source tarball is available for download from sourceforge.net.
P.S. MERRY CHRISTMAS!
The full ChangeLog snippet goes like this:
liblicense 0.8.1 (2008-12-24):
* Cleanups in the test suite: test_predicate_rw’s path joiner finally works
* Tarball now includes data_empty.png
* Dynamic tests and static tests treat $HOME the same way
* Fix a major issue with requesting localized informational strings, namely that the first match would be returned rather than all matches (e.g., only the first license of a number of matching licenses). This fixes the Python bindings, which use localized strings.
* Add a cooked PDF example that actually works with exempi; explain why that is not a general solution (not all PDFs have XMP packets, and the XMP packet cannot be resized by libexempi)
* Add a test for writing license information to the XMP in a PNG
* Fix a typo in exempi.c
* Add basic support for storing LL_CREATOR in exempi.c
* In the case that the system locale is unset (therefore, is of value “C”), assume English
* Fix a bug with the TagLib module: some lists were not NULL-terminated
* Use calloc() instead of malloc()+memset() in read_license.c; this improves efficiency and closes a crasher on amd64
* Improve chooser_test.c so that it is not strict as to the *order* the results come back so long as they are the right licenses.
* To help diagnose possible xdg_mime errors, if we detect the hopeless application/octet-stream MIME type, fprintf a warning to stderr.
* Test that searching for unknown file types returns a NULL result rather than a segfault.
64 bit woes (almost) cleared up
Nathan Kinkade, August 2nd, 2008
As I mentioned in a recent post, we have upgraded our servers to 64 bit. All of them are now running amd64 for Debian. The first three server were upgraded remotely, but we noticed that a few applications were constantly dying due to segmentation faults. There was some speculation that this was a strange consequence of the remote upgrade process, so we upgraded the 4th server by reprovisioning it with Server Beach as a 64 bit system, cleanly installed from scratch.
Well, it turned out that even the cleanly installed 64 bit system was having problems. So I installed the GNU Debugger, which I had never actually used before. I attached it to one of the processes that was having a problem, and what should immediately reveal itself but:
(gdb) c
Continuing.
[New Thread 1090525536 (LWP 16948)]Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1082132832 (LWP 16865)]
0×00002aaaaacfcd91 in tidySetErrorSink () from /usr/lib/libtidy-0.99.so.0
Nathan Yergler made a few changes to cc.engine, the application that was having a problem, and which is based on Zope, to remove any dependencies to libtidy, and the segfaults ceased. We haven’t had the time to debug libtidy itself, but it would seem that there was some incompatibility between the version we had installed and a 64 bit system.
We are still having a problem with cgit segfaulting, and that is the next thing to look into … 1 down, 1 to go.
No Comments »liblicense 0.8 (important) fixes RDF predicate error
asheesh, July 30th, 2008
Brown paper bag release: liblicense claims that the RDF predicate for a file’s license is http://creativecommons.org/ns#License rather than http://creativecommons.org/ns#license. Only the latter is correct.
Any code compiled with liblicense between 0.6 and 0.7.1 (inclusive) contains this mistake.
This time I have audited the library for other insanities like the one fixed here, and there are none. Great thanks to Nathan Yergler for spotting this. I took this chance to change ll_write() and ll_read() to *NOT* take NULL as a valid predicate; this makes the implementation simpler (and more correct).
Sadly, I have bumped the API and ABI numbers accordingly. It’s available in SourceForge at http://sf.net/projects/cctools, and will be uploaded to Debian and Fedora shortly (and will follow from Debian to Ubuntu).
I’m going to head to Argentina for a vacation and Debconf shortly, so there’ll be no activity from on liblicense for a few weeks. I would love help with liblicense in the form of further unit tests. Let’s squash those bugs by just demonstrating all the cases the license should work in.
No Comments »32 to 64 bit remotely
Nathan Kinkade, July 15th, 2008
A couple months ago I posted here about some of our experiences with Varnish Cache as an HTTP accelerator. By and large I have been very impressed with Varnish. We even found that it had the unexpected benefit of acting as a buffer in front of Apache, preventing Apache from getting overwhelmed with too many slow requests. Apache would get wedged once it had reached it’s MaxClients limit, whereas Varnish seems to happily queue up thousands of requests even if the backend (Apache) is going slowly.
However, after a while we started running into other problems with Varnish, and I found the probable answer in a bug report at the Varnish site. It turns out that Varnish was written with a 64 bit system in mind. That isn’t to say that it won’t work nicely on a 32 bit system, just that you better not expect high server load, or else you’ll start running into resource limitations in a hurry. This left us with about 2 options: Move to 64 bit or ditch Varnish for something like Squid. Seeing as how I was loathe to do the latter, we decided to go 64 bit, which in any case is another logical step into the 21st century.
The problem was that our servers are co-located in data centers around the country. We didn’t want to hassle with reprovisioning all of the them. Asheesh did the the first remote conversion based on some outdated document he found on remotely converting from Red Hat Linux to Debian. It went well and we haven’t had a single problem on that converted machine since. Varnish loves 64bit.
I have now converted two more machines, and this last time I documented the steps I took. I post them here for future reference and with the hope that it may help someone else. Note that these steps are somewhat specific to Debian Linux, but the concepts should be generally applicable to any UNIX-like system. There are no real instructions below, so you just have to infer the method from the steps. See the aforementioned article for more verbose, though dated, explanations. BE WARNED that if you make a mistake and don’t have some lovely rescue method then you may be forced to call your hosting company to salvage the wreckage:
- [ssh server]
- aptitude install linux-image-amd64
- reboot
- [ssh server]
- sudo su -
- aptitude install debootstrap # if not already installed
- swapoff -a
- sfdisk -l /dev/sda # to determine swap partition, /dev/sda5 in this case
- mke2fs -j /dev/sda5
- mount /dev/sda5 /mnt
- cfdisk /dev/sda # set /dev/sda5 to type 83 (Linux)
- debootstrap –arch amd64 etch /mnt http://http.us.debian.org/debian
- mv /mnt/etc /mnt/etc.LOL
- cp -a /etc /mnt/
- mv /mnt/boot /mnt/boot.LOL
- cp -a /boot /mnt/ # this is really just so that the dpkg post-install hooks don’t issue lots of warnings about things not being in /boot that it expects.
- chroot /mnt
- aptitude update
- aptitude dist-upgrade
- aptitude install locales
- dpkg-reconfigure locales # optional (I selected All locales, default UTF-8)
- aptitude install ssh sudo grub vim # and any other things you want
- aptitude install linux-image-amd64
- vi /etc/fstab # change /dev/sda5 to mount on / and comment out old swap entry
- mkdir /home/nkinkade # just so I have a home, not necessary really
- exit # get out of chroot
- vi /boot/grub/menu.lst # change root= of default option from sda6 to sda5
- reboot
- [ssh server]
- sudo su -
- mount /dev/sda6 /mnt
- chroot mnt
- dpkg –get-selections > ia32_dpkg_selections
- exit
- mv /home /home.LOL
- cp -a /mnt/home /
- mv /root /root.LOL
- cp -a /mnt/root /
- mkdir /mnt/ia32
- mv /mnt/* /mnt/ia32
- mv /mnt/.* /mnt/ia32
- cp -a bin boot dev etc etc.LOL home initrd initrd.img lib lib64 media opt root sbin srv tmp usr var vmlinuz /mnt
- mkdir /mnt/proc /mnt/sys
- vi /mnt/etc/fstab # make /dev/sda6 be mounted on / again, leave swap commented out
- vi /boot/grub/menu.lst # change the default boot option back to root=/dev/sda6
- reboot
- [ssh server]
- sudo su -
- mkswap /dev/sda5
- vi /etc/fstab (uncomment swap line)
- swapon -a
- dpkg –set-selections < /ia32/ia32_dpkg_selections
- apt-get dselect-upgrade # step through all the questions about changed /etc/files, etc.
