Debian LTS report for February 2017

February 2017 was my sixth month as a Debian LTS team member. I was allocated 5 hours and had 9,75 hours left over from January 2017. This makes a total of 14,75 hours. I spent all of them doing the following:

  • DLA 831-1: Fix buffer overflows in gtk-vnc
  • Reviewed the apache2 2.2.22-13+deb7u8 upload, improved the patches
  • Reviewed CVE-2017-5666 (mp3splt)
  • DLA 836-1: Fix command injection vulnerability in munin cgi script
Posted Sat 25 Feb 2017 05:22:01 PM UTC Tags:

Debian LTS report for January 2017

January 2017 was my fifth month as a Debian LTS team member. I was allocated 12 hours and had 6,75 hours left over from December 2016. This makes a total of 18,75 hours. Unfortunately I found less time than expected to work on Debian LTS in January. In total, I spent 9 hours on the following security updates:

  • DLA 787-1: XSS protection via Content Security Policy for otrs2
  • DLA 788-1: fix vulnerability in pdns-recursor by dropping illegitimate long querys
  • DLA 798-1: fix multiple vulnerabilities in pdns
Posted Fri 10 Feb 2017 04:07:57 PM UTC Tags:

Debian LTS report for December 2016

December 2016 was my fourth month as a Debian LTS team member. I was allocated 12 hours. Unfortunately I turned out to have way less time for Debian and LTS work than expected, so I only spent 5,25 hours of them for the following tasks:

  • DLA 732-1: backported CSRF protection to monit 1:5.4-2+deb7u1
  • DLA 732-2: fix a regression introduced in last monit security update
  • DLA 732-3: fix another regression introduced in monit security update
  • Nagios3: port 3.4.1-3+deb7u2 and 3.4.1-3+deb7u3 updates to wheezy-backports
  • DLA-760-1: fix two reflected XSS vulnerabilities in spip
Posted Sun 08 Jan 2017 10:18:56 AM UTC Tags:

On CVE-2016-4484, a (security)? bug in the cryptsetup initramfs integration

On November 4, I was made aware of a security vulnerability in the integration of cryptsetup into initramfs. The vulnerability was discovered by security researchers Hector Marco and Ismael Ripoll of CyberSecurity UPV Research Group and got CVE-2016-4484 assigned.

In this post I'll try to reflect a bit on

What CVE-2016-4484 is all about

Basically, the vulnerability is about two separate but related issues:

1. Initramfs rescue shell considered harmful

The main topic that Hector Marco and Ismael Ripoll address in their publication is that Debian exits into a rescue shell in case of failure during initramfs, and that this can be triggered by entering a wrong password ~93 times in a row.

Indeed the Debian initramfs implementation as provided by initramfs-tools exits into a rescue shell (usually a busybox shell) after a defined amount of failed attempts to make the root filesystem available. The loop in question is in local_device_setup() at the local initramfs script

In general, this behaviour is considered as a feature: if the root device hasn't shown up after 30 rounds, the rescue shell is spawned to provide the local user/admin a way to debug and fix things herself.

Hector Marco and Ismael Ripoll argue that in special environments, e.g. on public computers with password protected BIOS/UEFI and bootloader, this opens an attack vector and needs to be regarded as a security vulnerability:

It is common to assume that once the attacker has physical access to the computer, the game is over. The attackers can do whatever they want. And although this was true 30 years ago, today it is not.

There are many "levels" of physical access. [...]

In order to protect the computer in these scenarios: the BIOS/UEFI has one or two passwords to protect the booting or the configuration menu; the GRUB also has the possibility to use multiple passwords to protect unauthorized operations.

And in the case of an encrypted system, the initrd shall block the maximum number of password trials and prevent the access to the computer in that case.

While Hector and Ismael have a valid point in that the rescue shell might open an additional attack vector in special setups, this is not true for the vast majority of Debian systems out there: in most cases a local attacker can alter the boot order, replace or add boot devices, modify boot options in the (GNU GRUB) bootloader menu or modify/replace arbitrary hardware parts.

The required scenario to make the initramfs rescue shell an additional attack vector is indeed very special: locked down hardware, password protected BIOS and bootloader but still local keyboard (or serial console) access are required at least.

Hector and Ismael argue that the default should be changed for enhanced security:

[...] But then Linux is used in more hostile environments, this helpful (but naive) recovery services shall not be the default option.

For the reasons explained about, I tend to disagree to Hectors and Ismaels opinion here. And after discussing this topic with several people I find my opinion reconfirmed: the Debian Security Team disputes the security impact of the issue and others agree.

But leaving the disputable opinion on a sane default aside, I don't think that the cryptsetup package is the right place to change the default, if at all. If you want added security by a locked down initramfs (i.e. no rescue shell spawned), then at least the bootloader (GNU GRUB) needs to be locked down by default as well.

To make it clear: if one wants to lock down the boot process, bootloader and initramfs should be locked down together. And the right place to do this would be the configurable behaviour of grub-mkconfig. Here, one can set a password for GRUB and the boot parameter 'panic=1' which disables the spawning of a rescue shell in initramfs.

But as mentioned, I don't agree that this would be sane defaults. The vast majority of Debian systems out there don't have any security added by locked down bootloader and initramfs and the benefit of a rescue shell for debugging purposes clearly outrivals the minor security impact in my opinion.

For the few setups which require the added security of a locked down bootloader and initramfs, we already have the relevant options documented in the Securing Debian Manual:

After discussing the topic with initramfs-tools maintainers today, Guilhem and me (the cryptsetup maintainers) finally decided to not change any defaults and just add a 'sleep 60' after the maximum allowed attempts were reached.

2. tries=n option ignored, local brute-force slightly cheaper

Apart from the issue of a rescue shell being spawned, Hector and Ismael also discovered a programming bug in the cryptsetup initramfs integration. This bug in the cryptroot initramfs local-top script allowed endless retries of passphrase input, ignoring the tries=n option of crypttab (and the default of 3). As a result, theoretically unlimited attempts to unlock encrypted disks were possible when processed during initramfs stage. The attack vector here was that local brute-force attacks are a bit cheaper. Instead of having to reboot after max tries were reached, one could go on trying passwords.

Even though efficient brute-force attacks are mitigated by the PBKDF2 implementation in cryptsetup, this clearly is a real bug.

The reason for the bug was twofold:

  • First, the condition in setup_mapping() responsible for making the function fail when the maximum amount of allowed attempts is reached, was never met:

    setup_mapping()
    {
      [...]
      # Try to get a satisfactory password $crypttries times
      count=0                              
    while [ $crypttries -le 0 ] || [ $count -lt $crypttries ]; do export CRYPTTAB_TRIED="$count" count=$(( $count + 1 )) [...] done if [ $crypttries -gt 0 ] && [ $count -gt $crypttries ]; then message "cryptsetup: maximum number of tries exceeded for $crypttarget" return 1 fi [...] }

    As one can see, the while loop stops when $count -lt $crypttries. Thus the second condition $count -gt $crypttries is never met. This can easily be fixed by decreasing $count by one in case of a successful unlock attempt along with changing the second condition to $count -ge $crypttries:

    setup_mapping()
    {
      [...]
      while [ $crypttries -le 0 ] || [ $count -lt $crypttries ]; do
          [...]
          # decrease $count by 1, apparently last try was successful.
          count=$(( $count - 1 ))
          [...]
      done
      if [ $crypttries -gt 0 ] && [ $count -ge $crypttries ]; then
          [...]
      fi
      [...]
    }
    

    Christian Lamparter already spotted this bug back in October 2011 and provided a (incomplete) patch, but back then I even managed to merge the patch in an improper way, making it even more useless: The patch by Christian forgot to decrease $count by one in case of a successful unlock attempt, resulting in warnings about maximum tries exceeded even for successful attemps in some circumstances. But instead of adding the decrease myself and keeping the (almost correct) condition $count -eq $crypttries for detection of exceeded maximum tries, I changed back the condition to the wrong original $count -gt $crypttries that again was never met. Apparently I didn't test the fix properly back then. I definitely should do better in future!

  • Second, back in December 2013, I added a cryptroot initramfs local-block script as suggested by Goswin von Brederlow in order to fix bug #678692. The purpose of the cryptroot initramfs local-block script is to invoke the cryptroot initramfs local-top script again and again in a loop. This is required to support complex block device stacks.

    In fact, the numberless options of stacked block devices are one of the biggest and most inglorious reasons that the cryptsetup initramfs integration scripts became so complex over the years. After all we need to support setups like rootfs on top of LVM with two separate encrypted PVs or rootfs on top of LVM on top of dm-crypt on top of MD raid.

    The problem with the local-block script is that exiting the setup_mapping() function merely triggers a new invocation of the very same function.

    The guys who discovered the bug suggested a simple and good solution to this bug: When maximum attempts are detected (by second condition from above), the script sleeps for 60 seconds. This mitigates the brute-force attack options for local attackers - even rebooting after max attempts should be faster.

About disclosure, wording and clickbaiting

I'm happy that Hector and Ismael brought up the topic and made their argument about the security impacts of an initramfs rescue shell, even though I have to admit that I was rather astonished about the fact that they got a CVE assigned.

Nevertheless I'm very happy that they informed the Security Teams of Debian and Ubuntu prior to publishing their findings, which put me in the loop in turn. Also Hector and Ismael were open and responsive when it came to discussing their proposed fixes.

But unfortunately the way they advertised their finding was not very helpful. They announced a speech about this topic at the DeepSec 2016 in Vienna with the headline Abusing LUKS to Hack the System.

Honestly, this headline is missleading - if not wrong - in several ways:

  • First, the whole issue is not about LUKS, neither is it about cryptsetup itself. It's about Debians integration of cryptsetup into the initramfs, which is a compeletely different story.
  • Second, the term hack the system suggests that an exploit to break into the system is revealed. This is not true. The device encryption is not endangered at all.
  • Third - as shown above - very special prerequisites need to be met in order to make the mere existance of a LUKS encrypted device the relevant fact to be able to spawn a rescue shell during initramfs.

Unfortunately, the way this issue was published lead to even worse articles in the tech news press. Topics like Major security hole found in Cryptsetup script for LUKS disk encryption or Linux Flaw allows Root Shell During Boot-Up for LUKS Disk-Encrypted Systems suggest that a major security vulnerabilty was revealed and that it compromised the protection that cryptsetup respective LUKS offer.

If these articles/news did anything at all, then it was causing damage to the cryptsetup project, which is not affected by the whole issue at all.

After the cat was out of the bag, Marco and Ismael aggreed that the way the news picked up the issue was suboptimal, but I cannot fight the feeling that the over-exaggeration was partly intended and that clickbaiting is taking place here. That's a bit sad.

Posted Tue 06 Dec 2016 02:21:10 PM UTC Tags:

Debian LTS report for November 2016

Noevember 2016 was my third month as a Debian LTS team member. I was allocated 11 hours and had 1,75 hours left from October. This makes a total of 12,75 hours. In November I spent all 12,75 hours (and even a bit more) preparing security updates for spip, memcached and monit.

In particular, the updates of spip and monit took a lot of time (each one more than six hours). The patches for both packages were horrible to backport as the affected codebase changed a lot between the Wheezy versions and current upstream versions. Still it was great fun and I learned a lot during the backporting work. Due to the intrusive nature of the patches, I also did much more extensive testing before uploading the packages, which took quite a bit of time as well.

Monit 5.4-2+deb7u1 is not uploaded to wheezy-security yet as I decided to ask for further review and testing on the debian-lts mailinglist first.

Below follows the list of items I worked on in November in the well known format:

  • DLA 695-1: several XSS, CSRF and code execution flaws fixed in spip 2.1.17-1+deb7u6
  • DLA 701-1: integer overflows, buffer over-read fixed in memcached 1.4.13-0.2+deb7u2
  • CVE-2016-7067: backported CSRF protection to monit 5.4-2+deb7u1
Posted Wed 30 Nov 2016 07:30:18 PM UTC Tags:

Debian LTS Report for October 2016

October 2016 was my second month as a payed Debian LTS Team member. I was allocated 12 hours and spent 10,25 hours of them as follows:

Posted Mon 31 Oct 2016 10:25:06 PM UTC Tags:

Debian LTS Report for September 2016

September 2016 was my first month as a payed Debian LTS Team member. After doing two small uploads to wheezy-security in August and got to know the LTS Team workflow, this month I was allocated 9 hours by Freexian. I spent all 9 hours on working on security updates to Debian Wheezy.

In particular, I worked on the following issues:

  • DLA 612-1: libtomcrypt PKCS#1 RSA signature verification
  • DLA 617-1: libarchive out of bounds and denial of service
  • DLA 625-1: libcurl escape/unescape integer overflows
  • DLA 627-1: pdns qname's length>255b, missing zone size limits
  • worked on mat issue with embeded images in PDFs (#826101)

For reference, these were the issues I worked on in August:

  • DLA 584-1: libsys-syslog-perl opportunistic loading of optional modules
  • DLA 589-1: mupdf out of bounds write access to memory locations
Posted Sat 01 Oct 2016 08:34:53 AM UTC Tags:

Apache2: Rewrite REQUEST_URI based on a bulk list of GET parameters in QUERY_STRING

Recently I searched for a solution to rewrite a REQUEST_URI based on GET parameters in QUERY_STRING. To make it even more complicated, I had a list of ~2000 parameters that have to be rewritten like the following:

if %{QUERY_STRING} starts with one of <parameters>:
    rewrite %{REQUEST_URI} from /new/ to /old/

Honestly, it took me several hours to find a solution that was satisfying and scales well. Hopefully, this post will save time for others with the need for a similar solution.

Research and first attempt: RewriteCond %{QUERY_STRING} ...

After reading through some documentation, particularly Manipulating the Query String, the following ideas came to my mind at first:

RewriteCond %{REQUEST_URI} ^/new/
RewriteCond %{QUERY_STRING} ^(param1)(.*)$ [OR]
RewriteCond %{QUERY_STRING} ^(param2)(.*)$ [OR]
...
RewriteCond %{QUERY_STRING} ^(paramN)(.*)$
RewriteRule /new/ /old/?%1%2 [R,L]

or instead of an own RewriteCond for each parameter:

RewriteCond %{QUERY_STRING} ^(param1|param2|...|paramN)(.*)$

There has to be something smarter ...

But with ~2000 parameters to look up, neither of the solutions seemed particularly smart. Both scale really bad and probably it's rather heavy stuff for Apache to check ~2000 conditions for every ^/new/ request.

Instead I was searching for a solution to lookup a string from a compiled list of strings. RewriteMap seemed like it might be what I was searching for. I read the Apache2 RewriteMap documentation here and here and finally found a solution that worked as expected, with one limitation. But read on ...

The solution: RewriteMap and RewriteCond ${mapfile:%{QUERY_STRING}} ...

Finally, the solution was to use a RewriteMap with all parameters that shall be rewritten and check given parameters in the requests against this map within a RewriteCond. If the parameter matches, the simple RewriteRule applies.

For the inpatient, here's the rewrite magic from my VirtualHost configuration:

RewriteEngine On
RewriteMap RewriteParams "dbm:/tmp/rewrite-params.map"
RewriteCond %{REQUEST_URI} ^/new/
RewriteCond ${RewriteParams:%{QUERY_STRING}|NOT_FOUND} !=NOT_FOUND
RewriteRule ^/new/ /old/ [R,L]

A more detailed description of the solution

First, I created a RewriteMap at /tmp/rewrite-params.txt with all parameters to be rewritten. A RewriteMap requires two field per line, one with the origin and the other one with the replacement part. Since I use the RewriteMap merely for checking the condition, not for real string replacement, the second field doesn't matter to me. I ended up putting my parameters in both fields, but you could choose every random value for the second field:

/tmp/rewrite-params.txt:

param1 param1
param2 param2
...
paramN paramN

Then I created a DBM hash map file from that plain text map file, as DBM maps are indexed, while TXT maps are not. In other words: with big maps, DBM is a huge performance boost:

httxt2dbm -i /tmp/rewrite-params.txt -o /tmp/rewrite-params.map

Now, let's go through the VirtualHost configuration rewrite magic from above line by line. First line should be clear: it enables the Apache Rewrite Engine:

RewriteEngine On

Second line defines the RewriteMap that I created above. It contains the list of parameters to be rewritten:

RewriteMap RewriteParams "dbm:/tmp/rewrite-params.map"

The third line limits the rewrites to REQUEST_URIs that start with /new/. This is particularly required to prevent rewrite loops. Without that condition, queries that have been rewritten to /old/ would go through the rewrite again, resulting in an endless rewrite loop:

RewriteCond %{REQUEST_URI} ^/new/

The fourth line is the core condition: it checks whether QUERY_STRING (the GET parameters) is listed in the RewriteMap. A fallback value 'NOT_FOUND' is defined if the lookup didn't match. The condition is only true, if the lookup was successful and the QUERY_STRING was found within the map:

RewriteCond ${RewriteParams:%{QUERY_STRING}|NOT_FOUND} !=NOT_FOUND

The last line is a simple RewriteRule from /new/ to /old/. It is executed only if all previous conditions are met. The flags are R for redirect (issuing a HTTP redirect to browser) and L for last (causing mod_rewrite to stop processing immediately after that rule):

RewriteRule ^/new/ /old/ [R,L]

Known issues

A big limitation of this solution (compared to the ones above) is, that it looks up the whole QUERY_STRING in RewriteMap. Therefore, it works only if param is the only GET parameter. In case of additional GET parameters, the second rewrite condition fails and nothing is rewritten even if the first GET parameter is listed in RewriteMap.

If anyone comes up with a solution to this limitation, I would be glad to learn about it :)

Posted Sat 17 Sep 2016 03:52:59 PM UTC Tags:

Data recovery with ddrescue, testdisk and sleuthkit

From time to time I need to recover data from disks. Reasons can be broken flash/hard disks as well as accidently deleted files. Fortunately, this doesn't happen to often, which on the downside means that I usually don't remember the details about best practice.

Now that a good friend asked me to recover very important data from a broken flash disk, I take the opportunity to write down what I did and hopefully don't need to read the same docs again next time :)

Disclaimer: I didn't take the time to read through full documentation. This is rather a brief summary of the best practice to my knowledge, not a sophisticated and detailed explanation of data recovery techniques.

Create image with ddrescue

First and most secure rule for recovery tasks: don't work on the original, use a copied image instead. This way you can do, whatever you want without risking further data loss.

The perfect tool for this is GNU ddrescue. Contrary to dd, it doesn't reiterate over a broken sector with I/O errors again and again while copying. Instead, it remembers the broken sector for later and goes on to the next sector first. That way, all sectors that can be read without errors are copied first. This is particularly important as every extra attempt to read a broken sector can further damage the source device, causing even more data loss.

In Debian, ddrescue is available in the gddrescue package:

apt-get install gddrescue

Copying the raw disk content to an image with ddrescue is as easy as:

ddrescue /dev/disk disk-backup.img disk.log

Giving a logfile as third argument has the great advantage that you can interupt ddrescue at any time and continue the copy process later, possibly with different options.

In case of very large disks where only the first part was in use, it might be useful to start with copying the beginning only:

ddrescue -i0 -s20MiB /dev/disk disk-backup.img disk.log

In case of errors after the first run, you should start ddrescue again with direct read access (-d) and tell it to try again bad sectors three times (-r3):

ddrescue -d -r3 /dev/disk disk-backup.img disk.log

If some sectors are still missing afterwards, it might help to run ddrescue with infinite retries for some time (e.g. one night):

ddrescue -d -r-1  /dev/disk disk-backup.img disk.log

Inspect the image

Now that you have an image of the raw disk, you can take a first look at what it contains. If ddrescue was able to recover all sectors, chances are high that no further magic is required and all data is there.

If the raw disk (used to) contain a partition table, take a first look with mmls from sleuthkit:

mmls disk-backup.img

In case of a intact partition table, you can try to create device maps with kpartx after setting up a loop device for the image file:

losetup /dev/loop0 disk-backup.img
kpartx -a /dev/loop0

If kpartx finds partitions, they will be made available at /dev/mapper/loop0p1, /dev/mapper/loop0p2 and so on.

Search for filesystems on the partitions with fsstat from sleuthkit on the partition device map:

fsstat /dev/mapper/loop0p1

Or directly on the image file with the offset discovered by mmls earlier. This also might work in case of

fsstat -o 8064 disk-backup.img

The offset obviously is not needed if the image contains a partition dump (without partition table):

fsstat disk-backup.img

In case that a filesystem if found, simply try to mount it:

mount -t <fstype> -o ro /dev/mapper/loop0p1 /mnt

or

losetup -o 8064 /dev/loop1 disk-backup.img
mount -t <fstype> -o ro /dev/loop1 /mnt

Recover partition table

If the partition table is broken, try to recover it with testdisk. But first, create a second copy of the image, as you will alter it now:

ddrescue disk-backup.img disk-backup2.img
testdisk disk-backup2.img

In testdisk, select a media (e.g. Disk disk-backup2.img) and proceed, then select the partition table type (usually Intel or EFI GPT) and analyze -> quick search. If partitions are found, select one or more and write the partition structure to disk.

Recover files

Finally, let's try to recover the actual files from the image.

testdisk

If the partition table recovery was sucessfull, try to undelete files from within testdisk. Go back to the main menu and select advanced -> undelete.

photorec

Another option is to use the photorec tool that comes with testdisk. It searches the image for known file structures directly, ignoring possible filesystems:

photorec sdb2.img

You have to select either a particular partition or the whole disk, a file system (ext2/ext3 vs. other) and a destination for recovered files.

Last time, photorec was my last resort as the fat32 filesystem was so damaged that testdisk detected only an empty filesystem.

sleuthkit

sleuthkit also ships with tools to undelete files. I tried fls and icat. fls searches for and lists files and directories in the image, searching for parts of the former filesystem. icat copies the files by their inode numer. Last time I tried, fls and icat didn't recover any new files compared to photorec.

Still, for the sake of completeness, I document what I did. First, I invoked fls in order to search for files:

fls -f fat32 -o 8064 -pr disk-backup.img

Then, I tried to backup one particular file from the list:

icat -f fat32 -o 8064 <INODE>

Finally, I used the recoup.pl script from Dave Henk in order to batch-recover all discovered files:

wget http://davehenk.googlepages.com/recoup.pl
chmod +x recoup.pl
vim recoup.pl
[...]
my $fullpath="~/recovery/sleuthkit/";
my $FLS="/usr/bin/fls";
my @FLS_OPT=("-f","fat32","-o","8064","-pr","-m $fullpath","-s 0");
my $FLS_IMG="~/recovery/disk-image.img";
my $ICAT_LOG="~/recovery/icat.log";
my $ICAT="/usr/bin/icat";
my @ICAT_OPT=("-f","fat32","-o","8064");
[...]

Further down, the double quotes around $fullfile needed to be replaced by single quotes (at least in my case, as $fullfile contained a subdir called '$OrphanFiles'):

system("$ICAT @ICAT_OPT $ICAT_IMG $inode > \'$fullfile\' 2>> $ICAT_LOG") if ($inode != 0);

That's it for now. Feel free to comment with suggestions on how to further improve the process of recovering data from broken disks.

Posted Fri 19 Aug 2016 12:34:21 AM UTC Tags:

Hello world!

Finally, this is my first post to my own blog. For years, I refrained from having an own blog, even though I think about it since 10 years at least.

Today I decided to give it a try, let's see where it takes me. I'll write about Free Software topics from time to time, probably with a strong emphasis on Debian GNU/Linux.

Welcome to everybody who steps by.

Cheers,
mejo

Posted Thu 18 Aug 2016 03:46:53 PM UTC Tags:

This blog is powered by ikiwiki.