Category: itunes & ipod

After I wrote all this mess, I decided to break it out from the regular day-to-day stuff that makes up the bulk of sounds familiar, just to spare the typical audience from something they likely wouldn’t be interested in.  But for those who may get down with the music/audio/technical junk, here’s some.

Well folks, I thought I had it all.  A while back when I got the new 2009 Beatles remastered catalog in lossless FLAC format I figured I had the best sounding version of the stuff to be had by humans. I was even super happy that I was able to losslessly transcode the FLAC files into Apple’s own lossless audio format for pristine playback on the iPod (I know, no comments about my 6G iPod Classic’s crappy DAC or sound quality in general here, please).  Anyway, put short – I was pleased that, on my portable device, I had the best-sounding versions of the Beatles’ records you could get.

Then EMI (not Apple, and the subtle distinction is important here for Beatlemaniacs) goes ahead and releases the entire catalog again in digital format, via a special USB key, and this time they offer a “24 bit”  lossless FLAC option.  What does this mean?  I dunno… not a lot unless you want me to get way technical… oh wait, you do?  If you wanna skip the techy stuff, move forward to the very last paragraph here (“Goodnight”) and just know that the 24 bit version is, for some scientific reason, better than what I had previously.  For the smarty-pantses out there…

Regular readers tuned-out?  Gone?  Good.

See, normal, CD-quality tunes have a bit-depth of 16 bits per sample (a sample being sized as 1/44,100th of second).  Higher-definition audio, such as DVD-quality audio, often comes at at an increased bit-depth of 24 bits per sample, and a smaller sample size (sometimes 48 kHz, sometimes 96 kHz, sometimes as much 192 kHz).  If you’re nerdy, you can think of it as an analog-to-digital thing, trying to take enough discrete points of digital data to accurately represent a continual analog sound.  The more often you “record” the analog sound, and the higher “resolution” in which you make the recording can be thought of, respectively, as sample-rate and bit-depth.  (And for the real nerds those last two links are to a couple far-better explained Wikipedia articles on the subject.)

Anyway, EMI’s straight-to-digital release upped the Beatles catalog ante by offering the lossless files in 24 bit format (albeit still sampled at 44.1 kHz, which some maintain is akin to a cliffhanger ending in a summer blockbuster… leaving open the door for yet another “upgrade” to a higher quality version of the set at a later date).  Oh great; this now means that my heretofore “best quality” 16 bit files on the iPod are, in fact, now trumped.  And while folks will argue with me about an iPod’s ability to, with any noticeable difference in sound quality, reproduce 24 bit audio… I of course want to get these new recordings on my trusty portable device.

Problem is getting 24-bit FLAC files converted over to 24-bit ALAC (m4a) files – using Windows – hasn’t historically been the easiest thing in the world. At this point in time, it’s rather trivial if you’re in the MacOS environment, but quite a bit more tricky in a Windows environment (especially if you prefer open-source stuff or you’re not overly-anxious to spend money on a pay-for piece of software to do the task).  If you’re a Mac person, you’ve got XLD or Max at your side and’ll have hardly any issues getting high-def stuff onto your iPods (provided you’re stubborn like me and want to do it regardless of whether or not the playback quality warrants it).  If you’re a Windows user, you’re options are significantly more narrow.

However, I’m here to report my personal success in the hopes that others out there may share in it.  See, just last week the venerable application dBpoweramp released a version of their great conversion software that supports encoding of 24 bit Apple lossless files.  What’s more, the software does the conversion in a batch format, from the original directories, in mere seconds, and it’s completely free for a 28 day trial period – no strings attached.  Wonderful.  And, as a check, the final files imported into iTunes show 24 bit / 44,100 Hz as expected (and I’m sure would match the sampling rate on a 96 kHz file as well).  Losslessly perfect!

For what it’s worth, as long as you can get files encoded into an Apple container, an iPod classic will indeed support 24 bit playback (I have the 6th generation 160GB, your mileage may vary).  And, even with the internet, it’s pretty hard to nail down the sampling-rate limitations… but from experience I can tell you my 6G classic has no issues at all with the 24 bit / 44.1 kHz Beatles ALACs.  Apparently, however, the iPod does have a sampling-rate ceiling of 48 kHz, and reportedly trying to load anything sampled at a higher rate than this onto your iPod will cause iTunes to spit it right back (yet my Apple TV is able to play 24 bit / 96 kHz files via the optical output, which is fantastic).

So for me, I’ll ignore the audiophile mantra of “you’ll never be able to tell on the iPod anyway,” delete my old 16 bit ALAC files from the iPod, and go with my newly-FLAC-transcoded 24 bit / 44.1 kHz ALAC versions of the Beatles’ catalog for portability.  I mean, even amongst all the stuff I read online about the quality of the iPod’s DAC, there’s reason to believe you might be able to tell a difference.  But at home, I’ll try loading the ALACs onto the Apple TV and playing them back via optical to the surround system for the “best” listening… (since I don’t want to have to buy one of those mega-expensive iPod DAC bypassing docks/transports anytime soon…).  Anyway, I’ve said it before: I’m no audiophile.  I obsess about it, but, when it comes down to it, I can’t hear the difference.

Anyway, if you’re a Windows user and dBpoweramp worked for you, you’ll likely end up like me and buy it despite doing all you needed to do with it long before the free trial expires… I mean, it’s worth it  – and down the road when more stuff comes in higher-definition format you’re gonna want to be able to re-encode on the fly anyway.  Drop me a line if you had luck with it, or go buy it… whatever.

I got some Beatles to go listen to…

Goodnight.

Are you like me in that you see some advantages to using iTunes for managing your music collection and synchronizing it with your various Apple devices, but are completely underwhelmed by its basic functionality?  If so, this entry may be able to help you.  If not, then… well… not.

I have a huge collection of music on disk (~300GB) that I maintain manually and am very anal about.  I like the way my files are organized, I like being able to reorganize and re-tag them whenever I want with my choice of 3rd party tools (Tag&Rename, The Godfather, Windows), and I have an extremely anal methodology for pre-processing and merging newly acquired files (files not acquired via iTunes)  to the “master” location.

However, despite my hands-on tendencies when it comes to controlling my digital music collection, there have been several times in the past when I’ve wished I had my entire library present in iTunes (all explain why a little better below).

For a while now I’ve wondered if there’s an “easy” way to have the best of both worlds.  To have my music “in” iTunes while not giving up my low-level control of the actual files.  To be able to work with my media through whatever means I desire and not have these things interfere with each other.

Turns out there was a solution, at least for me, although the “easy” part might be subjective (I’m willing to go to great lengths to manage my music the way I want to, you may be less so after reading this diatribe).  Through the following I’ll document my journey to playing nice with iTunes, and hopefully this might help some other anal music collector looking to do something similar.

Let’s start off by documenting my situation:

My intended goal:

Start using iTunes as the default “library manager” for all my music.

Why do I want to do this?:

There are several advantages, as I see it, to having iTunes manage my music.  All of these are based on the fact that I’m using Apple devices (iPod Classic & iPhone 3G) to actually listen to my music (the benefits of iTunes as a library manager kinda drop off if that’s not the case).  Some of the advantages:

• When you manually manage music on your iPod sans iTunes, any metadata generated on the iPod is not tracked back into your master on-disk library.  If I have my library in iTunes, however,  using the “sync” option means metadata generated from listening habits on the device is now “backed up” into my iTunes library file and I won’t lose it should I upgrade to a new device (more on that below).
• I also like that by taking advantage of the “sync” feature, I can “backup” things like smart playlists, which I currently have only on my iPod (since they make no sense in an iTunes that has no library, as mine now stands).
• Again, using the sync feature means that any metadata changes I make to my files on disk are automatically carried over to the files on my iPod.  When you manually manage adding music to your iPod/iPhone, you lose this “connection” to your master on-disk library.
• If I buy a new iPod, I can simply plug it in and re-synchronize my entire library with a click (including that oh-so-important-to-me track-level metadata I mentioned above).  When you’re manually managing music on the device, this is not an option.  So, if your iPod dies or gets lost, you have to rebuild the collection on it by dragging and dropping everything back.  Painful.  This is probably the single most important thing to me, as I’ve gone through it before.

Success criteria:

1. I don’t want the physical structure of my files on disk to be modified in any way.  I don’t want anything renamed, reorganized, re-anything.  I want my file structure on disk to remain completely unchanged through this big iTunes import.
2. I want to retain “manual” control of my files on disk.  This means having  the ability to rename, reorganize, re-tag, move, delete, etc. these files using applications/methods other than iTunes.  I want to be able to do this without messing up my iTunes library (creating “orphan” entries, out-of-sync pointers, etc.).
3. In addition to the manual control I’d like to maintain in #2 above, I’d also like to be able to use iTunes to manage files if and when I feel like it.
4. If, per #2 above, I’ve manually deleted, moved, or changes files using a 3rd party application or Windows (in other words, not using iTunes) I need iTunes to be “smart” enough to sync up with what’s now on-disk.  In other words, I want my iTunes library to always mirror my manually-controlled library on-disk.
5. Since I don’t get my music from the iTunes store, I also need iTunes to be “smart” about watching my master folder on disk for newly-added files.  When it finds something new, I expect it to be added to the library.   Again, in other words, I want my iTunes library to always mirror my manually-controlled library on-disk.

I know, it may seem picky.  But that’s what I want.

Resolution:

So, was I able to meet all these criteria above and successfully “merge” my music-management process with an iTunes-managed library?  I sure was.  For those interested, I’ll walk through the criteria and let you know how I achieved each one to my own satisfaction:

1. iTunes actually plays pretty nicely here, allowing you to uncheck the “Keep iTunes music folder organized” and “Copy files to iTunes music folder when adding to library” options under Preferences|Advanced (see image below).
   For me, I needed to uncheck both of these (the former to satisfy my criteria #1, the latter to satisfy my criteria #2).  Criteria met.
2. Unfortunately, when you use iTunes to manage your music – iTunes assumes it’s the lone player in this game.  You can go modify files on-disk to your heart’s content, but none of these changes will be reflected in iTunes unless you re-add the modified files to your library.  This is the main problem I have with iTunes, it refuses to acknowledge you might do things to your music outside of iTunes… I did, however, manage to meet this criteria, for the how – see “the panacea” statement below.
3. This one is easy if I can meet 2, 4, and 5.  All I need is for iTunes and my manual processes not to step on each other.  This is more a requirement of my manual process than of iTunes.
4. Again unfortunately, Apple expects that if you’re adding or deleting music, you’re doing it via iTunes.  It makes no accommodations for music file management via other means.  For how I solved this, see “the panacea” below.
5. Again again unfortunately, Apple refuses to accept the fact that people might get their music from anywhere other than the iTunes store, and iTunes is incapable of monitoring folders in this way.  Fortunately, there are some 3rd party applications that can do this for you –  See “the panacea” below.

The panacea: How I met criteria 2-5:

So, how’d I do it?  Believe it or not, there are a couple free pieces of software out there on the great wide internet that can accomplish these things for you (and me).  iTunes Library Updater and iTunes Folder Watch.  Both of them do essentially the same thing, but I’ll tell you below why I chose the former of the two  as my solution.

First up, what do these applications do?  Simply, they allow me to force iTunes to monitor a folder or multiple folders on disk for changes, and synchronize my iTunes library to the folder(s) (without modifying the files themselves on disk).  While this is exactly what I want, there is one major difference between the way each operates.

While both applications keep track of newly added or removed files in your watched folder(s), only iTunes Library Updater includes the ability to monitor track-level metadata (ID3 tag) changes as a synchronization criteria.  For instance, if you’ve used a 3rd-party application (Tag&Rename, etc.) to change some tag data or add album artwork to your MP3s on disk, iTLU will notice this, and update the files for you in iTunes.  iTunes Folder Watch does not include this level of functionality.  So, if you change ID3 information outside of iTunes, you have to acutally play or “get info” on the updated files in your iTunes library to get a refreshed version of the tag information.

For this reason, I consider iTunes Library Updater the better solution, and will continue explaining my “panacea” in the context of using it rather than iTunes Folder Watch.

To make this whole thing work, you simply configure iTLU with the folders you want to be automatically monitored and mirrored into your iTunes library (along with several handy advanced criteria regarding the actual sync process) and hit “go.”  And, since iTLU comes with a command-line interface, the whole process can be scripted into a batch file that you can add to your Windows Scheduled Tasks (I won’t write detailed instructions for this, as someone’s already done a great job over here).  It’s in this way that I have my iTunes library automatically synchronized to my master on-disk library every night while I sleep.

Before you move along here, I would be remiss if I didn’t note the fact that the iTunes Library Updater application is no longer being actively developed.  In fact, it hasn’t been updated since something like iTunes version 6.  So, the fact that it continues to work with the current version of iTunes (versions 8.x as of writing) is pretty much gravy.  Bottom line, future changes to iTunes could completely break iTunes Library Updater at any time. This is a risk I am taking.

My thoughts are that if and when this day comes, I will simply have to switch to iTunes Folder Watch, wait for Apple to implement in iTunes, break down and code something myself, or give up on iTunes for good.

OK, but what’s the process?:

Wow, you actually read this far?  I’m shocked you’re still with me.  If you’re way down here, you probably have a good idea how I did my whole migration, but just to make it simple I’ll sum it up in some easy-to-follow steps for you:

1. Download and install iTunes Library Updater.
2. Configure iTLU to point to the music folders you desire it to mirror into your iTunes library, and check whichever options you want.
3. Save your configuration as an iTLU “profile” to ease repeated future use.
4. Hit “Start” in iTLU and crack a beer as the application imports everything into your iTunes library.
5. Make a batch file that invokes iTLU using the profile you saved in step 3 above (using the console version’s /p switch), and add this batch file to your Windows Scheduled Tasks as often as you’d like (I do it every single night).
6. Continue enjoying your Godlike control over your digital music files while simultaneously taking advantage of the benefits that having your collection in an iTunes library brings.

Conclusion:

And, as for getting your music into iTunes, that’s it.  If you’re satisfied then I think you can stop reading now.  But, because I tend to write a lot… I actually have a few more points to make that, if you’ve come this far, you may be interested in reading.

If you’re done with me here, I appreciate you hanging in there.  If you’ve got comments or suggestions (particularly if you’ve got some super-simple “better way” than what I’ve outlined here), please don’t hesitate to drop a comment below and let me know.  Hope it helped, take care.

Some further notes (for the really anal):

First, a big postscript to all you other “manually manage” people out there.  I know you, you’re just like me.  As you read this, there are exactly zero files in your iTunes library.  You drag-and-drop everything you need right onto your iPod and only use iTunes as the facilitator for that.  But, if you’re an advanced listener like me, you might also have some on-iPod smart playlists defined that do super-fancy stuff with the tunes you’ve manually added to your beloved device.   And, one disadvantage of the “convert to iTunes” method I’ve outlined above is that you won’t be able to import any metadata that’s been generated on your device (through your regular listening) into your new iTunes-based library.

You may be thinking it would be nice to “retrieve” this data before bringing your on-disk collection into iTunes, so that you can continue to build data around your listening habits and drive your smart playlists.  And, you’re right, it totally would be nice to do this.  I, however, made a big concession on this point when doing my huge “gonna start using iTunes” decision.

See, I too, have always said that if I were to make the conversion to managing my entire library via iTunes I’d like to be able to preserve all the metadata I’ve accumulated listening to my manually-added tunes on the iPod (meaning I’d like to be able to keep things like “play count,” “skip count,” “last played,” etc.).   And, you actually can do this.  Yes – you totally can.  See, you can use a piece of software called CopyTrans to first grab all the music off your iPod and into iTunes.  As it does this, it will copy over your precious metadata too (it works, I tried it).

The only issue with this is the extra overhead it adds to the above process…  See if you’re a “manual manager” like me, once you’ve extracted the files (and their associated metadata) off your iPod and into iTunes (which also means they physically went somewhere on disk), you’re going to have duplicate copies of those files in your master collection.  And, since the metadata you want does not “live” inside the MP3 tag data itself, but rather in the iTunes library XML file – you’ll now need to somehow get rid of those metatdata-less dupes in your mater collection (in favor of the metadata-associated versions now living on-disk from the CopyTrans import).

Your head hurt yet?

To try and simplify the explanation, you now want to keep the iPod-extracted files and remove the duplicate files from your master collection.  This is a pain, especially if you have a 160GB iPod Classic like me.  Oh, it’s doable… it really is, and you could keep and carry-over all your on-iPod generated metadata… but it’s just too much work.  So, in my case, I just decided that my big conversion to iTunes would be accompanied by a loss of all my on-iPod metadata.  That simple.  Kinda crappy, but better, in my opinion, than the time-consuming multi-step alternative outlined above.

All part of the paradigm shift, I figure.

Second, I can’t help but assume that, if you’re reading this article and your digital music collection means this much to you that you may also be facing another problem I’ve worked through.  Specifically, now that you’ve got your entire collection into iTunes, how the heck do you manage what actually goes on your iPod and what doesn’t?  It’s the classic “my iPod is too small for my music” problem.  I’ll try and address that in a future entry, because it’s just too long to work through were it tacked on here.

That’s it!  Thanks for reading, take care!

A middle-blog before my typical midnight post, dedicated to some tech content. Move along if that’s not your thing.

If you’ve read some of my banal iPod-related ramblings here before, you’ll know that I’m a big fan of using iTunes/iPod Smart Playlists to configure interesting musical selections. One of my favorite Smart Playlists I have on the ‘Pod is the “Unheard” list. A simple playlist that grabs all items where the “playcount” equals zero, and theoretically eases the task of making sure I’ve heard all those gems lurking in the back corners of my disk. The only problem with the playlist, and, in fact, with any shuffle-based playlist, is that it gets skewed heavy towards artists that are better represented on the iPod. Now look, I don’t need a lesson in statistics here, OK? I realize that, if out of 500 tracks on my iPod, 250 are the Grateful Dead, I’m going to see the Dead pop up pretty often in a true shuffle (as would be the case with my randomly-picked “unheard” list).

Problem is, I actually want to have every single Dead Dick’s Picks album on my iPod, just on the off chance that I can impress some Deadhead by saying “You bet I do” when they ask, passing the bong, “Hey man, do you have that ’75 Berkely gig, you know, the one where Donna Jean couldn’t hit the high notes in ‘Rain and Snow?'” And, I want each of the twenty-nine takes it took The Beatles to get “Hold Me Tight” right, not to mention all fifteen live versions of “Over the Hills and Far Away” Zeppelin performed on their ’73 US tour. I really do want to have all that on my iPod, all the time. I don’t want, however, the thousands and thousands of songs that pepper my iPod as a result of that fanaticism to “overpower” all the other stuff when shuffling. Here’s where you say, “Too bad Dave, you can’t have it both ways.”

Oh but I can! Here’s how I managed to limit the number of tracks per artist in a shuffled playlist.

First, make a smart playlist of all your music minus the overpopulated artists. I did this based on the catch-all criteria of track-time being greater than zero, and then filtered out the Grateful Dead, Beatles, and Led Zeppelin (my three most heavily populated artists). You could just as easily do it basing it off the iPod’s default “Music” playlist (which contains your entire library). However you do it, what you should end up with is a playlist containing your entire collection minus your most heavily represented artists.

Next, make a separate playlist for each of your overpopulated artists, limiting the number of songs to a reasonable number (I chose one hundred) chosen at random. Do this by using the “artist is” criteria along with the “limit to” filter. In my case, this means I have three playlists: Beatles, Zeppelin, and the Dead, each limited to one-hundred songs chosen at random from the thousands available for each artist.

Finally, create a new playlist that pulls music from the playlists you just made in the previous steps (you can use “in playlist” as a criteria for a playlist). You’ll need to make sure that you set the match criteria to “any” instead of “all” on that last one, or you’ll get a playlist with zero items. This newest playlist is essentially your entire collection, including your overpopulated artists, but limiting them to one-hundred (or a number of your choosing) tracks each. And, from now forward, instead of basing all your shuffle-themed playlists around the iPod’s default “Music” playlist, you can base them off of your new limited-representation list. Voila!

Postscript: If you do create sub-lists such as my one-hundred item random ones described above, you may notice that, over time, these playlists are not magically “refreshed” with new random tunes via iTunes. Despite the more-than-somewhat misleading name, “live updating” does not mean the playlist will choose a new batch of random songs, it means only that, when you add more songs to the iPod/iTunes that fit the smart playlist criteria, they’ll be accounted for and captured. If you’re looking to get some form of “auto refreshing” for your random tune selection (as I was, makes things more interesting), you’ll need to add some further elimination criteria to the playlist. I chose to add a criteria that “last played” is “not within the last” one week. This way, once a song is filtered into the playlist and you’ve heard it recently, it’s eliminated from the playlist and replaced with another (per the “limit to XX tracks” tickbox). Anyway, hope that helps.

You can likely think of all sorts of other limited-shuffle tricks you can do with playlist-combining, which makes using Smart Playlists a fun way to experience your music in different ways. Too bad Apple hasn’t added a way to “hide” certain Smart Playlists from showing on the iPod. It would be neat to be able to mask out the ones that are only “supporting” lists as building blocks to a final one (like the hundred-track ones required as interim input to create the final list above). Maybe with a new firmware, eh?

See ya!