Wayne and Layne

Convert M4A into MP3 with GNU/Linux, recursive and multithreaded

On a similar thread as an old post, Convert M4A into MP3 with GNU/Linux, I needed to convert a bunch of iTunes M4A unprotected audio files into standard MP3 files. With iTunes’ fondness for sorting by artist first, then by album, I ended up with a ton of single-file folders, making any scripting a bit of a pain. Also, converting a bunch of M4A files to MP3 is something that should be done in parallel on a multi-core CPU.

Here’s a snippet using find and xargs‘ multi-threading support to recursively find all M4A files and convert them to MP3 with as many threads as you want.

First, here’s the convm4a script that is called from the find/xargs line. Call it with a single argument, the location of an M4A file. The script with change to that directory and convert the file to a similarly-named MP3 file. Put this somewhere on your PATH (possibly in ~/bin/) and be sure to chmod +x convm4a.
#!/bin/bash

DIRNAME=`dirname "$1"`
FILENAME=`basename "$1" .m4a`
#echo "$DIRNAME"
cd "$DIRNAME"
faad -o - "${FILENAME}.m4a" | lame - "${FILENAME}.mp3" &> /dev/null

Test out that script to make sure it works. Then, run this line from the top of the directory full of your M4A files:

find -name '*\.m4a' -type f -print0 | xargs -P 4 -n 1 -0 convm4a

The -P 4 means to use four concurrent executions of convm4a, and the -n 1 means to only pass one argument to each invocation of convm4a. Change the number 4 to reflect the number of threads you want to run. You probably don’t want to exceed the number of CPU cores in your computer.

Cross-platform animation of still frames with MEncoder

Many times I’ve needed to take a series of still frames and create a video file to view them as an animation. Sometimes the images come from a simulation, and sometimes the images are from a video editing project where I had to edit individual frames. Regardless, I’ve spent a while searching for the best magic codes to provide to MEncoder to produce a universally compatible output video. I’ve tested video files created this way and they worked in QuickTime on OSX 10.6, Totem on Ubuntu Linux 9.10 (with the ubuntu-restricted-extras codec pack installed), and Windows XP with the K-Lite codec pack installed. Let me know if files produced this way don’t work for your system.

Since MEncoder’s default and preferred container format is AVI, I used that instead of MPEG. I used the default mpeg4 code, which should be a recent DivX or XviD codec, widely available on all common operating systems. I used MP3 encoding for the audio, sourced from a standalone .wav file created separately for this animation.

mencoder mf://*.png -mf fps=25:type=png -ovc lavc -lavcopts vcodec=mpeg4 -audiofile audio.wav -oac mp3lame -o output_video.avi

Build an AVR-GCC Toolchain

The main benefit of using Atmel’s line of AVR microcontrollers is the well established avr-gcc toolchain, based on the free and open source gcc compiler. There is a nicely packaged distribution for Windows, but nothing nice for Linux users. Given that a compiler toolchain is fairly important, I thought I should try building my own. Reproduced below are my notes from installing all the components of the avr-gcc toolchain.

Click through for a detailed guide on how to download, compile, and install the necessary tools to get an avr-gcc toolchain going with Linux. Also includes a link to download the completed package to save yourself the time of building the toolchain from scratch, if you want.

http://www.mbeckler.org/microcontrollers/avrgcc_build/

Easily capture a background process’s PID

If you’ve ever worked with the dd command to move data around in bulk, you know that to keep tabs on the data transfer you need to send a signal to it’s process ID (PID). One easy way to capture the PID is to use the Bash shell’s secret code $! which is the PID of the last job sent to run in the background. This is also useful for capturing a job’s PID programatically when spawning background jobs from a script.

$ dd if=/dev/hda of=/dev/hdb &
$ pid=$!

Then, you can use the pid variable to work with that job. Here, with dd we can ask it how far the transfer is towards completion:

$ kill -USR1 $pid
1337+0 records in
1337+0 records out

Controlled Parallel Processing with Bash

OMG parallel processing! If you’re not interested in teaching yourself the glory and torture that is pthreads, but still have a lot of jobs to run, you might be interested in this. Frequently I need to do explorations of a design space by tuning different parameters and running a simulation, which makes for a lot of simulation runs. With all the multi-core machines available nowadays, it’s nice to run lots of jobs in parallel to reduce the waiting time, but you don’t want to run more jobs than available cores. This script will allow you to run as many jobs in parallel as you want, and starting the next job as soon as a running job finishes.

#!/bin/bash

# put the first character in square brackets,
# to prevent the grep command from showing up
# in the process list with the same string as the program
PROGNAME_REGEX="[m]y_prog"

# set this equal to the number of cores, or however many
# jobs you want to run concurently
MAXJOBS="4"

for i in `seq 1 10`; do
    # while we are running the max number of jobs, wait here
    while [ `ps aux | grep $PROGNAME_REGEX | wc -l` -ge $MAXJOBS ]; do
        sleep 1
    done

    # You can use the parameter $i to change the behavior or
    # settings of your executing program
    # The & at the end starts this program in the background
    ./my_prog $i &
done

This might not work as well if your program’s name is something generic like, say, a.out, or MATLAB, so you might have to be more creative in the line that check how many jobs are running. Maybe change the ps command to only include processes belonging to your username, or do something fancy with the grep line.

Python version of seq for OSX

OSX is based on BSD, so it is missing some of the standard GNU utilities we’ve come to expect on a linux-y system. The most frequently used of these missing utilities is the seq utility, which is used to define a range of numbers, very useful in quick loops from the command line:

$ seq 6 12
6
7
8
9
10
11
12

You can also use the -w flag to get zero-padded output:

$ seq -w 6 12
06
07
08
09
10
11
12

The nearest BSD equivalent is something called jot which of course has more functionality but is not compatible with good old seq. I wrote a quick and dirty python script to emulate the simple behaviors of seq, specifically the base counting behavior and the -w behavior. Put it somewhere on your path (I’d suggest ~/bin/) and you should be good to go:

#!/usr/bin/env python

import sys, math

if len(sys.argv) < 3:
    print "Usage: %s [-w] min max" % sys.argv[0]
    sys.exit(1)

try:
    if len(sys.argv) == 4:
        wide = True
        min = int(sys.argv[2])
        max = int(sys.argv[3])
    else:
        wide = False
        min = int(sys.argv[1])
        max = int(sys.argv[2])
except:
    print "Error: min and max must be integers"
    sys.exit(1)

width = int(math.ceil(math.log10(max)))

for i in range(min, max + 1):
    if wide:
        print "%0*d" % (width, i)
    else:
        print "%d" % i

Record a ShoutCast Stream

In my hometown of Eau Claire, WI, there is a very cool community radio station called WHYS 96.3 FM. Since I don’t live there anymore, I was pleased to see that they are streaming their programming using ShoutCast. There is a radio show on Monday evenings that I want to listen to, but I have other obligations scheduled during that time, so I had to come up with a way to record the stream to disk, and listen to it later. I created a short script that can be run from cron to record an audio stream for a set duration to an mp3 file on disk. Click through for more details:

Save an Audio Stream to Disk (mbeckler.org)

SVG Circuit Symbols

When I need to draw up a circuit schematic, but don’t need or want to use a full-blown schematic capture program (Eagle, OrCad, KiCad, etc), I like to use Inkscape to draw simple circuits. I’ve collected a bunch of frequenly-used circuit symbols and standardized them in SVG format.

Available for download in high-quality vector SVG format at:

http://www.mbeckler.org/inkscape/circuit_symbols/

Tactile Metronome release!

Layne and I have been working on some microcontroller kits for folks to solder up and have some fun with. We’re ready to release the first one, and we’re sure proud of it! It’s a Tactile Metronome. What that means is it’s a little gadget you can tap and it will beep back at you the same rhythm. It shows you the BPM and has tempo adjustment buttons. A really cool part is that you can tap patterns into it and it will tap those back to you as well. We’ve worked really hard on the documentation, and Layne made some really amazing pictures showing how some of the trickier things work in the firmware.

The entire kit is open sourced. More details (and a place to order one!) are over at http://wayneandlayne.com/metronome.

Using command line arguments in Python in IDLE.

I’m not a big fan of integrated development environments (IDEs), but they can be nice to help people get introduced to a new language, or even to programming in general. Processing and Arduino wouldn’t be as easy for new people to use if they didn’t have a decent IDE.

Most useful scripts accept command line arguments to configure their behavior, but sometimes it is difficult to set command line parameters from within an IDE. I was recently working on a python project with a friend living in Florida, new to the world of programming, so I showed him how to use Python’s IDLE interface, a nice IDE for Python work. Unfortunately, there is no way to set command line arguments from IDLE, so our scripts weren’t working in the IDE. I found a way to manually set the command line arguments if a Python script is run from IDLE, but doesn’t interfere if the script is run from the console.

To use, insert this code right after the start of your main-line code, right before you do your parsing and validation of the input parameters. Be sure that the specified arguments below match what your program is expecting.


try:
    __file__
except:
    sys.argv = [sys.argv[0], 'argument1', 'argument2', 'argument2']