os x

Console flooding, USB timeouts & Garmin ANT+ by Paulo Fierro

While I was working this morning I ran into some issues with an app and went to check if the Console (under /Applications/Utilities) was reporting anything.

Unfortunately the log was filled with some USB device timing out with the following error:

AppleUSBEHCI::Found a transaction past the completion deadline on bus 0xfa, timing out! (Addr: 5, EP: 1)

A very noisy log

This timeout was annoyingly happening every few seconds. The error did give me a few clues. It was happening on USB bus 0xfa on a device attached to address 5.

To find the device in question I generated a System Report ( > About this Mac > More Info > System Report) and looked for the USB bus with the same address.

Finding the USB bus number

Finding the USB bus number

Then it was just a question of going through the devices connected to this bus that had the address in question.

The address of the USB device

The address of the USB device

Found you! The device in question was the Garmin ANT+ stick that transmits data wirelessly from my Garmin Forerunner 610 watch.

Unplugging it fixed the problem and the Console stopped being flooded.

fry.jpg

Setting a photo's creation date to the date it was taken by Paulo Fierro

First a little back story.

The other day my mom called me up saying that her 11" MacBook Air was out of space and running slow wondering what we could do about it. Knowing that OS X runs pretty terribly without some free space I figured I'd run some of the Onyx cleanup scripts, check out where the space was being used up with Daisy Disk and get rid of some of the crap.

Onyx cleared a few gigs, and Daisy Disk said that iPhoto was the main culprit — the library being nearly 80GB,  67% of the total space on the disk.

Firing up iPhoto I created a Smart Album to see how much space videos were taking up.

The Smart Album conditions

Turns out there was nearly 10GB in videos, so I selected all of the items in the Movies album and exported them to an external disk. After removing these from the library, emptying the iPhoto trash and the actual Trash the disk had about 15GB free.

Success!

Not so fast

Of course not. After handing the laptop back my mom complained that iPhoto was running really sluggish. I got it back, checked it out and it was unusable. Scrolling had a 5-6 second lag and the UI responsiveness was generally pretty terrible.

My guess is that removing those videos screwed up the library somehow, so I opened up iPhoto holding down CMD+OPT to get the repair menu up and ran each item on the list (repairing permissions, rebuilding thumbnails, reindexing photos, repairing the library). Many, many hours later everything was still terrible so I decided to start from scratch.

My mom's iPhoto library is fairly large — nearly 15,000 photos. Initially I thought her 11" may not be able to handle that many photos so I figured exporting all of the photos and only importing the ones from the last two years would be a way of seeing if that was the case.

I left the laptop overnight exporting the 15,000 photos, backed up the library and deleted the original. Now I just had to find the photos taken within the last two years.

What should be easy...

Once again foiled. All of the exported photos had date of creation and modification set to yesterday. The sub-folders were all album names and not really indicative of when the photos were taken. I peeked inside the iPhoto library package contents to the Original and Modified folders where the photos live but found duplicates, terrible names and general cruft so I figured the actual exported photos had to be salvaged somehow.

Then I remembered EXIF data. Every photo you take contains metadata about what camera was used to take it, the model, ISO, exposure time, etc. One of those fields is the date it was taken. So if I could figure out how to read the date taken I could write a script to "fix" the photos.

ExifTool to the rescue!

With ExifTool in hand I went off to Automator and wrote some Ruby that would loop through a set of folders and change the date modified to the date the photo was taken as dictated by its EXIF data. It looked like so (download it here):

I tested this on a folder and it worked! I could follow along what was going on in Console.app and all was good.

Then I ran it on the main folder and encountered an interesting bug:

Run Shell Script failed - too many arguments.

Automator was passing in the path names to all of the photos to Ruby as arguments, but someone could only handle 4096 of these. Well below the 15,000 items I wanted to process.

Having spent enough time on this already I decided to be lazy and just run it a few times. Due to the logging to the Console I could see how far it had got and simply resume it from there with a next batch. There's also a bug where photos with single quotes or ampersands had to be renamed to be processed, which I could have fixed but again, lazy.

Now only the modified date was supposed to be changed, but I guess OS X doesn't allow a creation date to be later than its modified date so they both changed as a bonus.

Now for some Spotlight magic...

Now that the photos finally had their dates back I could make a simple search for files that were JPEGs and created in the last two years.

Importing these into a new iPhoto library took a while but once it was done scrolling was fluid and the app was generally responsive. Great! So I backed up this library and decided to try importing the rest of the photos.

A few hours later all of the photos were back in iPhoto and the app was still scrolling smoothly. What the fuck?

My guess is that removing the videos screwed up the original library in a way that iPhoto itself couldn't fix. Maybe deleting the video files themselves from the disk would have been better — leaving orphans in the iPhoto library. I have no idea.

In any case it was a learning experience and yet again proof that Automator is truly a hidden gem that I don't use enough.

You can download the Automator app (open it in Automator to edit it if you like), but remember you must have ExifTool installed first.

Changing dates with Automator by Paulo Fierro

For my birthday Niqui got me this dive mask from Liquid Image. Its a nice mask and the video taken with it is great but the four rechargeable AAA batteries power it for about 90 minutes. Most of our dives are an hour or less so that's not a problem, and the mask comes with a battery recharger – however while charging the batteries the camera forgets the date.

You have to set the date each and every time. Its a minor thing but it annoys me so I don't do it. For some reason the default date is set to Monday, January 8, 2035 – a strange choice.

So because I'm lazy and don't change the date on the camera every time, when I eventually drag the video files into Final Cut I get crap like this:

Screen Shot 2012-10-15 at 11.53.34.png

Having some events with the right date and some in the future makes organising video for projects a nightmare. So after googling around I found that I could use the touch command in the Terminal to change the creation date. If I do this on each file before importing it into Final Cut I avoid the problem. So the command is basically:

touch -t yyyymmddHHMM <PATH_TO_FILE>

Still, doing this on multiple files is a massive pain so I toyed with the idea of writing a script for it. I have used Automator in the past and figured doing things like selecting items using the Finder's File Selection dialog is easier than using a command line script so that's what I did and you can see the "recipe" below.

It accepts a string which is the date you want the file to have been created, you select the files and you're done. It keeps the original time too, which may not be something you want but I don't really care about the time. As long as the files have the right date and are sequenced correctly its good enough for me.

You can download the application here. You can also open it up with Automator and modify it as you see fit.

While doing this I learned Automator actually has support for variables – who knew? Its a powerful tool and I don't think I use it enough.

Screen Shot 2012-10-15 at 12.00.36.png

Setting up a fresh Lion for Ruby development by Paulo Fierro

Update (July 30, 2012):

If you've upgraded to Mountain Lion the below instructions still work. Once complete you should check out this guide. It contains a tip on installing gcc via brew in order to avoid using CLANG which can result in occasional errors when installing/updating gems.

--- End of update ---

A few weeks back I decided to take the plunge and wipe my machine and set it up from scratch. The main reason being I was running out of space on my SSD and I just wanted to see what a fresh Lion install would be like.

Let me tell you, its FAST! Totally worth it. It did take me some time to find the right resources and shave some yaks so this is more of a guide for future-me to save some time. If you do carry on reading from this point I assume you know what you're doing :)

So now that everything is wiped and fresh and new (backed up of course) the main task is setting up ye old dev environment. Lets start at the beginning.

Xcode

Head over to the Mac App Store and download Xcode. Once you install it you'll find that it no longer lives in /Developer. Its been moved to /Applications and its command line tools are no longer in the $PATH on the command line.

We don't have to add them ourselves, simply start up Xcode, go to Preferences and then Downloads > Components. Click on the "Install" button next to "Command Line Tools".

Heroku

If you're deploying stuff on Heroku, grab the toolbelt here.

Homebrew

Previously I was using MacPorts, but after hearing so many great things about Homebrew I took the plunge. Its simply a great package manager and you can download it here. For example, OS X doesn't ship with wget, so to install it you can simply run the following on the command line:

brew install wget

Done. Moving on.

RVM

RVM is a command-line tool which allows you to easily install, manage, and work with multiple ruby environments from interpreters to sets of gems.

What it says on the tin. It allows you to easily install multiple versions of Ruby on your system. It also allows you create gem sets and swap between them. To install:

bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

Then:

source ~/.rvm/scripts/rvm

Now we're ready to install Ruby! The only problem is that with Xcode 4 the provided compiler (gcc) is LLVM based and not yet fully supported by Ruby and gems. Now before you go hunting for an older gcc (which I did) its not necessary.

To install Ruby 1.9.2 (which is required for Heroku) enter:

rvm install 1.9.2 --with-gcc=clang

When that is done, lets check what version of Ruby we have set as the system default:

ruby --version

Mine said Ruby 1.8.7. Lets set 1.9.2 to be default:

rvm use 1.9.2 --default

If you run ruby --version it should now say 1.9.2. If you ever want to go back to the system default simply enter rvm system. So now we have an easy way of swapping Ruby versions and you can read more about it here.

Sublime Text

Sublime Text is my favourite editor of all time. Get it here. Read some great tips and tricks here.

MySQL

Previously I was using MAMP. However I had an issue installing the do_mysql gem which is required for dm-mysql-adapter I was using with Datamapper.

Long story short I found the easiest thing to do was install mysql via brew instead of using MAMP. So to start off:

brew install mysql

Next to configure the installation (all one line):

mysql_install_db --verbose --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp

To get it to run on startup, first we have to create the directory if it doesn't exist:

mkdir -p ~/Library/LaunchAgents

Then we copy the mysql.plist over (the version number may have changed when you're reading this):

cp /usr/local/Cellar/mysql/5.5.20/homebrew.mxcl.mysql.plist ~/Library/LaunchAgents/

Then we tell launchctl to load it on startup:

launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

Now to set the password:

mysqladmin -u root password {new-password}

Phew! Ok, now mysql is installed and running on port 3306. To create your first database simply enter:

mysqladmin -u root -p create {your-database-name}

And some gem love...

Finally I recommend using bundler for gem management, foreman to run your app (especially if you're on Heroku), rerun to rerun your app when something changes, datamapper for ORM and sinatra for happiness.

Good luck!