If the legacy system has a sane, secure solution then you can implement a custom encryptor for Devise. However if that’s not the case or if you want to migrate to the Devise default way-of-life following convention over configuration and reducing maintenance headaches in the future, you should convert the passwords so that Devise can use them.

Since hopefully you stored hashes of your user’s passwords instead of encrypted or plaintext, the only moment you can retrieve the plaintext password for rehashing is when the user tries to log in.  At that point we’ll convert it to Devise. This way when a user logs in his legacy password will be automatically converted to the new system.

First, add the legacy_password boolean attribute to User. This is to mark the passwords that have already been converted. Make sure to set it to true for the existing (legacy) records.

$ rails g migration AddLegacyPasswordToUser legacy_password:boolean
      invoke  active_record
      create    db/migrate/20120508083355_add_legacy_password_to_users.rb
$ rake db:migrate

Devise creates the method User#valid_password? to check the password. Let’s override it and add our legacy password check or fall through to the default valid_password? if the password was already converted.  This code assumes the old password is stored in the same field as the converted password — encrypted_password.

class User < ActiveRecord::Base
 
...
 
  def valid_password?(password)
    if self.legacy_password?
      # Use Devise's secure_compare to avoid timing attacks
      if Devise.secure_compare(self.encrypted_password, User.legacy_password(password))
 
        self.password = password
        self.password_confirmation = password
        self.legacy_password = false
        self.save!
 
      else
        return false
      end
    end
 
    super(password)
  end
 
  # Put your legacy password hashing method here
  def self.legacy_password(password)
    return Digest::MD5.hexdigest("#{password}-salty-herring");
  end
end

Supporting two different systems like this could soon become a maintenance nightmare After a couple of months, when the most active users have logged in at least once and converted their passwords, you can easily remove the extra code. Legacy users will be forced to reset their password since it won’t match anymore. Using the legacy_user attribute you could send them a targeted mail or alert them of the fact that they won’t be able to log in anymore with their old password.

This example is based on polling and unfair: waiting processes are not guaranteed to be served on a first-come, first-served basis.

redis-semaphore

Using the pretty cool BLPOP command I’ve created the simple redis-semaphore gem which implements a blocking and fair semaphore and mutex. Here’s some pseudocode to explain how it works:

if GETSET(my_mutex_start, 1) != 1
  LPUSH(my_mutex, token)

BLPOP(my_mutex)
# Work here
work()
LPUSH(my_mutex, token)

The first two lines make sure the mutex is initialized only once. Using the atomic GETSET operation we set the key my_mutex_start to 1, and check to see if it already was 1 before we set it. If it was then another process already initialized the mutex (or is busy doing so). If it wasn’t, it’s up to us to initialize it: we push an element “token” to the my_mutex list.

This token is the key to the mutex: the process that removes the token from the list has locked the mutex. If the token is back on the list, the mutex is unlocked and another process can claim it and start working. This is accomplished in the next three lines.

BLPOP tries to pop the token off the my_mutex list. If the list is empty, it will block until another process pushes the token on it. Redis puts multiple BLPOP’s in a FIFO queue, so whoever calls BLPOP first, will be the first to actually get the token once it arrives.

Once the BLPOP returns, it means we have the token so we can do some work. Afterwards we push the token back on the list, unlocking the mutex.

Creating a semaphore instead of a mutex is as simple as pushing multiple tokens on the list during initialization. If you push on five then five processes will be able to lock the semaphore at a time.

user.name = “Rob”

This regular assignment is the most common and easiest to use. It is the default write accessor generated by Rails. The name attribute will be marked as dirty and the change will not be sent to the database yet.

You can undo the change by calling reload! or save the change to the database by calling save.

user.write_attribute(:name, “Rob”)

This is the method that is called by the default accessor above. A synonym for this function is user[:name] = “Rob”. It also has a read_attribute counterpart.

Just like above, this method does not yet change the attribute in the database. Use this method anywhere you need to bypass the default write accessor above, for example when you want to write a custom attribute= writer.

user.update_attribute(:name, “Rob”)

This method will change the attribute in the model and pass it straight to the database, without running any validations.

Two gotchas:

• Any other changed attributes are also saved to the database.
• Validations are skipped so you could end up with invalid data.

Because of that last quirk it’s a good practice to use update_attributes instead.

user.attributes = {:name => “Rob”}

This method will set all the attributes you pass it, except those who are protected from mass assignment if you’re using attr_protected or attr_accessible. The changes are not saved to the database.

You can override the mass assignment protection by passing false:

user.send(:attributes=, {:name => "Rob"}, false)

user.update_attributes(:name => “Rob”)

This method changes the attributes of the model, checks the validations, and updates the record in the database if it validates. Since it uses the above attributes= method, attributes protected from mass assignment are not changed.

Note that just like update_attribute this method also saves other changed attributes to the database.

Handy Cheat Sheet Table

If you want to understand more about these methods I suggest you check out their source code. Each time it’s only a couple of lines and it will really broaden your understanding of how Rails works!

1. Mass Assignment Protection for attributes= is overridable.

Since you use version control like all good developers, it’s possible these hardcoded credentials get committed. This poses a grave security risk, especially if you want to open source the code including the repository.

Here’s how to remove a password from any file, in all revisions, in a git repository:

$ git filter-branch --tree-filter "find . -type f -exec sed -i -e 's/originalpassword/newpassword/g' {} \;"

Just replace originalpassword with the word you want to replace, and newpassword with the word you want to replace it with¹.

After you’re done, you can check if your password really isn’t in any of the files anymore by grepping every revision²:

git filter-branch --tree-filter "grep -r originalpassword * || true"

If you’re positive the changes were done correctly, make sure to remove the automatically created backupfiles in refs/original/³.

Now enjoy a fine glass of wine, safe in the knowledge that your repository won’t reveal any of your secrets.

1. Here’s another handy one, deleting all the lines containing word.

   $ git filter-branch --tree-filter "find . -type f -exec sed -i -e '/$*word/d' {} \;"

2. By OR-ing with true we make sure the command is run in any revision, because if it returns false (e.g. originalpassword isn’t found in any of the files of a specific revision), git will think the filter failed and it won’t check the other revisions. You’ll notice if this happens since you’ll get a “tree filter failed” error on the first nonmatching revision.
3. If you don’t, sooner or later you’ll get the following error message:

   Cannot create a new backup.
   A previous backup already exists in refs/original/
   Force overwriting the backup with -f

Standard Sign Up Dialog

But maybe a password confirmation box would be useful for if you made a typo? Oh, not to worry! Right as you press the “Sign-Up” button, you get this security disaster:

“Let's render the previous user effort totally void!”

Your password, which you previously painstakingly entered blind, is now completely visible, right after you pressed Sign-Up. This obviously makes the previous masked password-box totally useless. The negative usability experience caused by the password-box was for nothing.

Of course they also send your password in an e-mail, just to make absolutely sure any bystander has seen your password, and if someone ever gets into your mail account, he or she also has access to your eToro account.

Here’s how to fix it, eToro:

• If you’re of the opinion that you should use a password inputbox, make sure the user’s password is never visible anywhere on your site. Any security you might have gained by masking the user’s password would be lost. Remember why password boxes were invented in the first place!
• Don’t send the user his password in an e-mail! If a bad person ever gains access to the user’s mail account, he also gains access to the user’s eToro account by searching for any mails containing the word “password”. Just make sure you have a “Password Recovery” feature which is both fast and secure to replace.

Here’s a sample application showing how:

#include <stdio.h>
#include <wordexp.h>
 
int main(int argc, char* argv[]) {
	wordexp_t exp_result;
	wordexp(arv[1], &exp_result, 0);
	printf(exp_result.we_wordv[0]);
}

That should pretty much tell you everything you need to know. Here are some of the results output by this app:

~/.vimrc becomes /home/david/.vimrc
.vimrc becomes .vimrc
~.vimrc becomes ~.vimrc
~blacky/.vimrc becomes /home/admin/blacky/.vimrc (blacky’s homedir is /home/admin/blacky)

As you can see, it handles pretty much every situation correctly.

Gmail Carbon Copy, or Gmailcc simply creates a back-up of your Gmail. It differs from existing alternatives because of two clever tricks: each mail is downloaded only once instead of once for every label while still saving the labels, and they’re stored in an actually usable, sparse Maildir format.

Gmail’s IMAP implementation is unique in that it maps labels to folders. The same mail will appear in different folders for every label attached to it. Regular IMAP clients like Thunderbird or getmail think each copy of the mail in a different folder is a new mail, and will download it again, even though it might just be a copy of a mail it downloaded earlier. Gmailcc detects “doubles” and will download each mail just once. Backups, especially the initial one, will finish much faster because of this and will take far less traffic.

Saving it in a usable Maildir format has the advantage that any regular mailserver like Courier can access your backup. It’s very practical: I’m using Gmailcc and Roundcube to access my mails on a webinterface if Gmail is down. It’s sparse because every mail is saved only once, while for every label a sizeless link is created instead of a true copy. This minimizes the space used to store the backup.

There are still some issues but it shouldn’t make your PC explode or kill your Gmail account. If you encounter bugs or would like to have features added, I encourage you to sign up and add a ticket.

Gmail Carbon Copy is open source (C++), licensed under the MIT license and works only on Linux at this time.

Can't locate object method "new" via package "LWP::UserAgent"
at /etc/munin/plugins/apache_processes line 152.
2009/07/08-17:00:02 Plugin "apache_processes" exited with status 512. ----
Can't locate object method "new" via package "LWP::UserAgent"
at /etc/munin/plugins/apache_accesses line 130.
2009/07/08-17:00:03 Plugin "apache_accesses" exited with status 512. ----
Can't locate object method "new" via package "LWP::UserAgent"
at /etc/munin/plugins/apache_volume line 130.
2009/07/08-17:00:03 Plugin "apache_volume" exited with status 512. ----

then the solution is to install the package libwww-perl which includes the required LWP:UserAgent package.

Make sure to restart munin-node afterwards:

$ /etc/init.d/munin-node restart

To hide your password from bystanders, innocent or otherwise.

That’s the one and only reason why we obscure the characters with stars. That co-worker sitting next to you, or the coffee-lady casually walking by, if it weren’t for the trusty password field they could have spotted and accidentally memorized your password while you’re entering it. It’s a great solution to a very real problem.

Redundancy at WordPress.com

All this is common knowledge, of course. So why am I repeating it? Because surprisingly, for most sites it’s redundant. All the websites out there that send your password by email, or show it when you’ve clicked the “activate account”-link are nullifying the sole reason of existence for the password field.

Since the user’s password is displayed on the screen in an e-mail, that coffee-lady can look at the password anyway. Worse: oft-times the user doesn’t know what’s coming when opening the mail or clicking the activation-link. He can’t pre-emptively check if anyone is in his vicinity before unknowingly revealing the password on his screen, which is an option when entering the password in a regular inputbox.

The conclusion is simple: if you think you can send the user his password by mail or show it in clear text on his profile, stop using the password inputbox. It won’t increase the level of security. By then, it only serves to annoy the user who has to enter his password blindly, twice even, possibly making an error along the way and having to try again. It’ll also tell the user the real degree of security you’re using, instead of fooling him with the asterisks.

(The real conclusion is of course to never show the password in cleartext, anywhere)

First of all, the methods explained in this series are not secure. Anyone with some low-level knowledge of filesystems can tell there’s hidden data when looking at a raw image of your disk. Always complement these methods using encryption and plausible deniability methods. TrueCrypt is an excellent way to do this.

Introduction

When a sector on a disk gets damaged, it becomes unusable.  Modern disks have spare sectors that are used to replace these bad sectors, so they’re handled and fixed automatically. If you’re young enough, you might never have witnessed these bad sectors, because modern hardware handles them transparently.

When the disk runs out of spare sectors, or never had any in the first place (like 3.5″ disks, or very old hard disks), the filesystem is the second line of defense. Inside the filesystem a list of known bad blocks—blocks on bad sectors—is stored. The filesystem takes care not to use these blocks and just skips them.

We can’t force the disk to remap certain blocks to spare sectors, but we can tell the filesystem which blocks have (supposedly) gone bad. If the blocks aren’t really damaged, any data we put there will never be touched, because the filesystem thinks it’s garbage anyway. That, is exactly what we’re going to do.

Practical

To keep it simple and fast, we’ll hide a whole partition inside a burst of bad blocks. The partition we’ll create has to be small and reside somewhere in the middle of the disk. We can’t put the partition at the beginning or the end of the disk, because most likely the filesystem requires an intact header at the start and end of the partition.

The partition has to be small enough to be able to fit inside the non-secret partition while not arousing suspicion. Some operating systems mark bad blocks as used blocks, which means if we put a 100MB partition inside bad blocks, the “parent” filesystem will always have at least 100MB in use. This could arouse suspicion when there aren’t any files on it.

I’ll be using my trusty 256MB Compactflash card for this, which is excellent for illustratory purposes.

Here’s what sfdisk has to say about it:

$ sudo sfdisk -l /dev/sde

Disk /dev/sde: 1009 cylinders, 9 heads, 56 sectors/track
Units = cylinders of 258048 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sde1          0       -       0          0    0  Empty
/dev/sde2          0       -       0          0    0  Empty
/dev/sde3          0       -       0          0    0  Empty
/dev/sde4          0       -       0          0    0  Empty

We can see the card is comprised of 1009 cylinders. I want to create a partition of about 20MB, which is about 82 cylinders on this disk (see the second line of sfdisk -l). Because we can’t create the partition at the start of the disk, let’s put it 214 cylinders in:

$ sudo sfdisk /dev/sde << EOF
214,82,6
EOF

Just like before, put FAT16 on it and transfer your secret data.

$ sudo mkfs.vfat -F16 /dev/sde1
mkfs.vfat 2.11 (12 Mar 2005)

If you want, you can copy the current partition table to the back of the disk for easy restoring, just like in the previous article.

Unmount it, and remove the partition:

$ sudo sfdisk /dev/sde << EOF
0,0,0
EOF

Now create the parent partition. This should at least encompass the whole secret partition. If you’ve copied the partition table to the back of the disk, make sure to leave at least the last cylinder free.

$ sudo sfdisk /dev/sde << EOF
,,6
EOF

Creating Bad Blocks

We need to calculate what blocks our secret partition resides on so we can mark them as bad. We know it starts at cylinder 214 and is 82 cylinders in size. Since on this disk, a cylinder is 258048 bytes big, the secret partition starts at byte 55222272. Divide this by the size of one block, which is 1024 bytes, and we get block 53928. Do the same for the size of the partition, and we find that 82 cylinders equal 20664 blocks. Now we know our partition starts at block 53928 and ends at block 74592. We’ll use a margin of 10 blocks on each side just in case our calculations aren’t precise.

Since we’re putting a FAT16 filesystem on it, we need to tell mkfs.vfat what blocks have supposedly gone bad. This is done by using a bad blocks file, which is a text-file with the address of each bad block on a new line. Let’s create our bad blocks file:

$ seq 53918 74602 > /tmp/badblocks

If you open /tmp/badblocks, you should see something like this:

53918
53919
53920
53921
...

To create the filesystem, we pass the bad blocks file using the -l parameter:

$ sudo mkfs.vfat -n "Camera" -l /tmp/badblocks /dev/sde1
mkdosfs 2.11 (12 Mar 2005)
20685 bad blocks

That’s it! You can now use your disk to your heart’s delight, nothing will touch your secret partition. One awesome way is to put the card in your camera and take some pictures with it. Your data will remain safe, and there’ll be nothing suspicious about a 4GB card “missing” some megabytes.

Revert

If you’ve smuggled your secret data across state borders, you’re ready to recover the secret partition. Just recreate the partition table to contain the secret partition:

$ sudo sfdisk /dev/sde << EOF
214,82,6
EOF

That’s it! You can even reuse the setup: by switching partition tables you’re effectively changing which partition is “active” on your card, and changing data in either partition won’t affect the other.

Advantages

• Pretty much undetectable
• Infinitely reusable
• Bad blocks are less suspicious than unallocated space

Disadvantages

• Quite complex to set up
• Possibly suspicious size discrepancy in empty filesystems