Beets

----------------------

The fifteenth (!) beta of beets is compendium of small fixes and features, most
of which represent long-standing requests. The improvements include matching
albums with extra tracks, per-disc track numbering in multi-disc albums, an
overhaul of the album art downloader, and robustness enhancements that should
keep beets running even when things go wrong. All these smaller changes should
help us focus on some larger changes coming before 1.0.

Please note that this release contains one backwards-incompatible change: album
art fetching, which was previously baked into the import workflow, is now
encapsulated in a plugin (the :doc:`/plugins/fetchart`). If you want to continue
fetching cover art for your music, enable this plugin after upgrading to beets
1.0b15.

* The autotagger can now find matches for albums when you have **extra tracks**
on your filesystem that aren't present in the MusicBrainz catalog. Previously,
if you tried to match album with 15 audio files but the MusicBrainz entry had
only 14 tracks, beets would ignore this match. Now, beets will show you
matches even when they are "too short" and indicate which tracks from your
disk are unmatched.
* Tracks on multi-disc albums can now be **numbered per-disc** instead of
per-album via the :ref:`per_disc_numbering` config option.
* The default output format for the ``beet list`` command is now configurable
via the :ref:`list_format_item` and :ref:`list_format_album` config options.
Thanks to Fabrice Laporte.
* Album **cover art fetching** is now encapsulated in the
:doc:`/plugins/fetchart`. Be sure to enable this plugin if you're using this
functionality. As a result of this new organization, the new plugin has gained
a few new features:

* "As-is" and non-autotagged imports can now have album art imported from
the local filesystem (although Web repositories are still not searched in
these cases).
* A new command, ``beet fetchart``, allows you to download album art
post-import. If you only want to fetch art manually, not automatically
during import, set the new plugin's ``autofetch`` option to ``no``.
* New album art sources have been added.

* Errors when communicating with MusicBrainz now log an error message instead of
halting the importer.
* Similarly, filesystem manipulation errors now print helpful error messages
instead of a messy traceback. They still interrupt beets, but they should now
be easier for users to understand. Tracebacks are still available in verbose
mode.
* New metadata fields for `artist credits`_: ``artist_credit`` and
``albumartist_credit`` can now contain release- and recording-specific
variations of the artist's name. See :ref:`itemfields`.
* Revamped the way beets handles concurrent database access to avoid
nondeterministic SQLite-related crashes when using the multithreaded importer.
On systems where SQLite was compiled without ``usleep(3)`` support,
multithreaded database access could cause an internal error (with the message
"database is locked"). This release synchronizes access to the database to
avoid internal SQLite contention, which should avoid this error.
* Plugins can now add parallel stages to the import pipeline. See
:ref:`writing-plugins`.
* Beets now prints out an error when you use an unrecognized field name in a
query: for example, when running ``beet ls -a artist:foo`` (because ``artist``
is an item-level field).
* New plugin events:

* ``import_task_choice`` is called after an import task has an action
assigned.
* ``import_task_files`` is called after a task's file manipulation has
finished (copying or moving files, writing metadata tags).
* ``library_opened`` is called when beets starts up and opens the library
database.

* :doc:`/plugins/lastgenre`: Fixed a problem where path formats containing
``$genre`` would use the old genre instead of the newly discovered one.
* Fix a crash when moving files to a Samba share.
* :doc:`/plugins/mpdupdate`: Fix TypeError crash (thanks to Philippe Mongeau).
* When re-importing files with ``import_copy`` enabled, only files inside the
library directory are moved. Files outside the library directory are still
copied. This solves a problem (introduced in 1.0b14) where beets could crash
after adding files to the library but before finishing copying them; during
the next import, the (external) files would be moved instead of copied.
* Artist sort names are now populated correctly for multi-artist tracks and
releases. (Previously, they only reflected the first artist.)
* When previewing changes during import, differences in track duration are now
shown as "2:50 vs. 3:10" rather than separated with ``->`` like track numbers.
This should clarify that beets isn't doing anything to modify lengths.
* Fix a problem with query-based path format matching where a field-qualified
pattern, like ``albumtype_soundtrack``, would match everything.
* :doc:`/plugins/chroma`: Fix matching with ambiguous Acoustids. Some Acoustids
are identified with multiple recordings; beets now considers any associated
recording a valid match. This should reduce some cases of errant track
reordering when using chroma.
* Fix the ID3 tag name for the catalog number field.
* :doc:`/plugins/chroma`: Fix occasional crash at end of fingerprint submission
and give more context to "failed fingerprint generation" errors.
* Interactive prompts are sent to stdout instead of stderr.
* :doc:`/plugins/embedart`: Fix crash when audio files are unreadable.
* :doc:`/plugins/bpd`: Fix crash when sockets disconnect (thanks to Matteo
Mecucci).
* Fix an assertion failure while importing with moving enabled when the file was
already at its destination.
* Fix Unicode values in the ``replace`` config option (thanks to Jakob Borg).
* Use a nicer error message when input is requested but stdin is closed.
* Fix errors on Windows for certain Unicode characters that can't be represented
in the MBCS encoding. This required a change to the way that paths are
represented in the database on Windows; if you find that beets' paths are out
of sync with your filesystem with this release, delete and recreate your
database with ``beet import -AWC /path/to/music``.
* Fix ``import`` with relative path arguments on Windows.

.. _artist credits: https://wiki.musicbrainz.org/Artist_Credit

---------------------

The centerpiece of this beets release is the graceful handling of
similarly-named albums. It's now possible to import two albums with the same
artist and title and to keep them from conflicting in the filesystem. Many other
awesome new features were contributed by the beets community, including regular
expression queries, artist sort names, moving files on import. There are three
new plugins: random song/album selection; MusicBrainz "collection" integration;
and a plugin for interoperability with other music library systems.

A million thanks to the (growing) beets community for making this a huge
release.

* The importer now gives you **choices when duplicates are detected**.
Previously, when beets found an existing album or item in your library
matching the metadata on a newly-imported one, it would just skip the new
music to avoid introducing duplicates into your library. Now, you have three
choices: skip the new music (the previous behavior), keep both, or remove the
old music. See the :ref:`guide-duplicates` section in the autotagging guide
for details.
* Beets can now avoid storing identically-named albums in the same directory.
The new ``%aunique{}`` template function, which is included in the default
path formats, ensures that Crystal Castles' albums will be placed into
different directories. See :ref:`aunique` for details.
* Beets queries can now use **regular expressions**. Use an additional ``:`` in
your query to enable regex matching. See :ref:`regex` for the full details.
Thanks to Matteo Mecucci.
* Artist **sort names** are now fetched from MusicBrainz. There are two new data
fields, ``artist_sort`` and ``albumartist_sort``, that contain sortable artist
names like "Beatles, The". These fields are also used to sort albums and items
when using the ``list`` command. Thanks to Paul Provost.
* Many other **new metadata fields** were added, including ASIN, label catalog
number, disc title, encoder, and MusicBrainz release group ID. For a full list
of fields, see :ref:`itemfields`.
* :doc:`/plugins/chroma`: A new command, ``beet submit``, will **submit
fingerprints** to the Acoustid database. Submitting your library helps
increase the coverage and accuracy of Acoustid fingerprinting. The Chromaprint
fingerprint and Acoustid ID are also now stored for all fingerprinted tracks.
This version of beets *requires* at least version 0.6 of `pyacoustid`_ for
fingerprinting to work.
* The importer can now **move files**. Previously, beets could only copy files
and delete the originals, which is inefficient if the source and destination
are on the same filesystem. Use the ``import_move`` configuration option and
see :doc:`/reference/config` for more details. Thanks to Domen Kožar.
* New :doc:`/plugins/random`: Randomly select albums and tracks from your library.
Thanks to Philippe Mongeau.
* The :doc:`/plugins/mbcollection` by Jeffrey Aylesworth was added to the core
beets distribution.
* New :doc:`/plugins/importfeeds`: Catalog imported files in ``m3u`` playlist
files or as symlinks for easy importing to other systems. Thanks to Fabrice
Laporte.
* The ``-f`` (output format) option to the ``beet list`` command can now contain
template functions as well as field references. Thanks to Steve Dougherty.
* A new command ``beet fields`` displays the available metadata fields (thanks
to Matteo Mecucci).
* The ``import`` command now has a ``--noincremental`` or ``-I`` flag to disable
incremental imports (thanks to Matteo Mecucci).
* When the autotagger fails to find a match, it now displays the number of
tracks on the album (to help you guess what might be going wrong) and a link
to the FAQ.
* The default filename character substitutions were changed to be more
conservative. The Windows "reserved characters" are substituted by default
even on Unix platforms (this causes less surprise when using Samba shares to
store music). To customize your character substitutions, see :ref:`the replace
config option <replace>`.
* :doc:`/plugins/lastgenre`: Added a "fallback" option when no suitable genre
can be found (thanks to Fabrice Laporte).
* :doc:`/plugins/rewrite`: Unicode rewriting rules are now allowed (thanks to
Nicolas Dietrich).
* Filename collisions are now avoided when moving album art.
* :doc:`/plugins/bpd`: Print messages to show when directory tree is being
constructed.
* :doc:`/plugins/bpd`: Use Gstreamer's ``playbin2`` element instead of the
deprecated ``playbin``.
* :doc:`/plugins/bpd`: Random and repeat modes are now supported (thanks to
Matteo Mecucci).
* :doc:`/plugins/bpd`: Listings are now sorted (thanks once again to Matteo
Mecucci).
* Filenames are normalized with Unicode Normal Form D (NFD) on Mac OS X and NFC
on all other platforms.
* Significant internal restructuring to avoid SQLite locking errors. As part of
these changes, the not-very-useful "save" plugin event has been removed.

.. _pyacoustid: https://github.com/beetbox/pyacoustid

refinements. A lyrics plugin is now included with beets; new audio properties
are catalogged; the ``list`` command has been made more powerful; the autotagger
is more tolerant of different tagging styles; and importing with original file
deletion now cleans up after itself more thoroughly. Many, many bugs—including
several crashers—were fixed. This release lays the foundation for more features
to come in the next couple of releases.

* The :doc:`/plugins/lyrics`, originally by `Peter Brunner`_, is revamped and
included with beets, making it easy to fetch **song lyrics**.
* Items now expose their audio **sample rate**, number of **channels**, and
**bits per sample** (bitdepth). See :doc:`/reference/pathformat` for a list of
all available audio properties. Thanks to Andrew Dunn.
* The ``beet list`` command now accepts a "format" argument that lets you **show
specific information about each album or track**. For example, run ``beet ls
-af '$album: $tracktotal' beatles`` to see how long each Beatles album is.
Thanks to Philippe Mongeau.
* The autotagger now tolerates tracks on multi-disc albums that are numbered
per-disc. For example, if track 24 on a release is the first track on the
second disc, then it is not penalized for having its track number set to 1
instead of 24.
* The autotagger sets the disc number and disc total fields on autotagged
albums.
* The autotagger now also tolerates tracks whose track artists tags are set
to "Various Artists".
* Terminal colors are now supported on Windows via `Colorama`_ (thanks to Karl).
* When previewing metadata differences, the importer now shows discrepancies in
track length.
* Importing with ``import_delete`` enabled now cleans up empty directories that
contained deleting imported music files.
* Similarly, ``import_delete`` now causes original album art imported from the
disk to be deleted.
* Plugin-supplied template values, such as those created by ``rewrite``, are now
properly sanitized (for example, ``AC/DC`` properly becomes ``AC_DC``).
* Filename extensions are now always lower-cased when copying and moving files.
* The ``inline`` plugin now prints a more comprehensible error when exceptions
occur in Python snippets.
* The ``replace`` configuration option can now remove characters entirely (in
addition to replacing them) if the special string ``<strip>`` is specified as
the replacement.
* New plugin API: plugins can now add fields to the MediaFile tag abstraction
layer. See :ref:`writing-plugins`.
* A reasonable error message is now shown when the import log file cannot be
opened.
* The import log file is now flushed and closed properly so that it can be used
to monitor import progress, even when the import crashes.
* Duplicate track matches are no longer shown when autotagging singletons.
* The ``chroma`` plugin now logs errors when fingerprinting fails.
* The ``lastgenre`` plugin suppresses more errors when dealing with the Last.fm
API.
* Fix a bug in the ``rewrite`` plugin that broke the use of multiple rules for
a single field.
* Fix a crash with non-ASCII characters in bytestring metadata fields (e.g.,
MusicBrainz IDs).
* Fix another crash with non-ASCII characters in the configuration paths.
* Fix a divide-by-zero crash on zero-length audio files.
* Fix a crash in the ``chroma`` plugin when the Acoustid database had no
recording associated with a fingerprint.
* Fix a crash when an autotagging with an artist or album containing "AND" or
"OR" (upper case).
* Fix an error in the ``rewrite`` and ``inline`` plugins when the corresponding
config sections did not exist.
* Fix bitrate estimation for AAC files whose headers are missing the relevant
data.
* Fix the ``list`` command in BPD (thanks to Simon Chopin).

.. _Colorama: https://pypi.python.org/pypi/colorama

-------------------------

This release focuses on making beets' path formatting vastly more powerful. It
adds a function syntax for transforming text. Via a new plugin, arbitrary Python
code can also be used to define new path format fields. Each path format
template can now be activated conditionally based on a query. Character set
substitutions are also now configurable.

In addition, beets avoids problematic filename conflicts by appending numbers to
filenames that would otherwise conflict. Three new plugins (``inline``,
``scrub``, and ``rewrite``) are included in this release.

* **Functions in path formats** provide a simple way to write complex file
naming rules: for example, ``%upper{%left{$artist,1}}`` will insert the
capitalized first letter of the track's artist. For more details, see
:doc:`/reference/pathformat`. If you're interested in adding your own template
functions via a plugin, see :ref:`writing-plugins`.
* Plugins can also now define new path *fields* in addition to functions.
* The new :doc:`/plugins/inline` lets you **use Python expressions to customize
path formats** by defining new fields in the config file.
* The configuration can **condition path formats based on queries**. That is,
you can write a path format that is only used if an item matches a given
query. (This supersedes the earlier functionality that only allowed
conditioning on album type; if you used this feature in a previous version,
you will need to replace, for example, ``soundtrack:`` with
``albumtype_soundtrack:``.) See :ref:`path-format-config`.
* **Filename substitutions are now configurable** via the ``replace`` config
value. You can choose which characters you think should be allowed in your
directory and music file names. See :doc:`/reference/config`.
* Beets now ensures that files have **unique filenames** by appending a number
to any filename that would otherwise conflict with an existing file.
* The new :doc:`/plugins/scrub` can remove extraneous metadata either manually
or automatically.
* The new :doc:`/plugins/rewrite` can canonicalize names for path formats.
* The autotagging heuristics have been tweaked in situations where the
MusicBrainz database did not contain track lengths. Previously, beets
penalized matches where this was the case, leading to situations where
seemingly good matches would have poor similarity. This penalty has been
removed.
* Fix an incompatibility in BPD with libmpc (the library that powers mpc and
ncmpc).
* Fix a crash when importing a partial match whose first track was missing.
* The ``lastgenre`` plugin now correctly writes discovered genres to imported
files (when tag-writing is enabled).
* Add a message when skipping directories during an incremental import.
* The default ignore settings now ignore all files beginning with a dot.
* Date values in path formats (``$year``, ``$month``, and ``$day``) are now
appropriately zero-padded.
* Removed the ``--path-format`` global flag for ``beet``.
* Removed the ``lastid`` plugin, which was deprecated in the previous version.

--------------------------

This version of beets focuses on transitioning the autotagger to the new version
of the MusicBrainz database (called NGS). This transition brings with it a
number of long-overdue improvements: most notably, predictable behavior when
tagging multi-disc albums and integration with the new `Acoustid`_ acoustic
fingerprinting technology.

The importer can also now tag *incomplete* albums when you're missing a few
tracks from a given release. Two other new plugins are also included with this
release: one for assigning genres and another for ReplayGain analysis.

* Beets now communicates with MusicBrainz via the new `Next Generation Schema`_
(NGS) service via `python-musicbrainzngs`_. The bindings are included with
this version of beets, but a future version will make them an external
dependency.
* The importer now detects **multi-disc albums** and tags them together. Using a
heuristic based on the names of directories, certain structures are classified
as multi-disc albums: for example, if a directory contains subdirectories
labeled "disc 1" and "disc 2", these subdirectories will be coalesced into a
single album for tagging.
* The new :doc:`/plugins/chroma` uses the `Acoustid`_ **open-source acoustic
fingerprinting** service. This replaces the old ``lastid`` plugin, which used
Last.fm fingerprinting and is now deprecated. Fingerprinting with this library
should be faster and more reliable.
* The importer can now perform **partial matches**. This means that, if you're
missing a few tracks from an album, beets can still tag the remaining tracks
as a single album. (Thanks to `Simon Chopin`_.)
* The new :doc:`/plugins/lastgenre` automatically **assigns genres to imported
albums** and items based on Last.fm tags and an internal whitelist. (Thanks to
`KraYmer`_.)
* The :doc:`/plugins/replaygain`, written by `Peter Brunner`_, has been merged
into the core beets distribution. Use it to analyze audio and **adjust
playback levels** in ReplayGain-aware music players.
* Albums are now tagged with their *original* release date rather than the date
of any reissue, remaster, "special edition", or the like.
* The config file and library databases are now given better names and locations
on Windows. Namely, both files now reside in ``%APPDATA%``; the config file is
named ``beetsconfig.ini`` and the database is called ``beetslibrary.blb``
(neither has a leading dot as on Unix). For backwards compatibility, beets
will check the old locations first.
* When entering an ID manually during tagging, beets now searches for anything
that looks like an MBID in the entered string. This means that full
MusicBrainz URLs now work as IDs at the prompt. (Thanks to derwin.)
* The importer now ignores certain "clutter" files like ``.AppleDouble``
directories and ``._*`` files. The list of ignored patterns is configurable
via the ``ignore`` setting; see :doc:`/reference/config`.
* The database now keeps track of files' modification times so that, during
an ``update``, unmodified files can be skipped. (Thanks to Jos van der Til.)
* The album art fetcher now uses `albumart.org`_ as a fallback when the Amazon
art downloader fails.
* A new ``timeout`` config value avoids database locking errors on slow systems.
* Fix a crash after using the "as Tracks" option during import.
* Fix a Unicode error when tagging items with missing titles.
* Fix a crash when the state file (``~/.beetsstate``) became emptied or
corrupted.

.. _KraYmer: https://github.com/KraYmer
.. _Next Generation Schema: https://musicbrainz.org/doc/XML_Web_Service/Version_2
.. _python-musicbrainzngs: https://github.com/alastair/python-musicbrainzngs
.. _acoustid: https://acoustid.org/
.. _Peter Brunner: https://github.com/Lugoues
.. _Simon Chopin: https://github.com/laarmen
.. _albumart.org: https://www.albumart.org/

---------------------------

This version of beets focuses on making it easier to manage your metadata
*after* you've imported it. A bumper crop of new commands has been added: a
manual tag editor (``modify``), a tool to pick up out-of-band deletions and
modifications (``update``), and functionality for moving and copying files
around (``move``). Furthermore, the concept of "re-importing" is new: you can
choose to re-run beets' advanced autotagger on any files you already have in
your library if you change your mind after you finish the initial import.

As a couple of added bonuses, imports can now automatically skip
previously-imported directories (with the ``-i`` flag) and there's an
:doc:`experimental Web interface </plugins/web>` to beets in a new standard
plugin.

* A new ``beet modify`` command enables **manual, command-line-based
modification** of music metadata. Pass it a query along with ``field=value``
pairs that specify the changes you want to make.

* A new ``beet update`` command updates the database to reflect **changes in the
on-disk metadata**. You can now use an external program to edit tags on files,
remove files and directories, etc., and then run ``beet update`` to make sure
your beets library is in sync. This will also rename files to reflect their
new metadata.

* A new ``beet move`` command can **copy or move files** into your library
directory or to another specified directory.

* When importing files that are already in the library database, the items are
no longer duplicated---instead, the library is updated to reflect the new
metadata. This way, the import command can be transparently used as a
**re-import**.

* Relatedly, the ``-L`` flag to the "import" command makes it take a query as
its argument instead of a list of directories. The matched albums (or items,
depending on the ``-s`` flag) are then re-imported.

* A new flag ``-i`` to the import command runs **incremental imports**, keeping
track of and skipping previously-imported directories. This has the effect of
making repeated import commands pick up only newly-added directories. The
``import_incremental`` config option makes this the default.

* When pruning directories, "clutter" files such as ``.DS_Store`` and
``Thumbs.db`` are ignored (and removed with otherwise-empty directories).

* The :doc:`/plugins/web` encapsulates a simple **Web-based GUI for beets**. The
current iteration can browse the library and play music in browsers that
support HTML5 Audio.

* When moving items that are part of an album, the album art implicitly moves
too.

* Files are no longer silently overwritten when moving and copying files.

* Handle exceptions thrown when running Mutagen.

* Fix a missing ``__future__`` import in ``embed art`` on Python 2.5.

* Fix ID3 and MPEG-4 tag names for the album-artist field.

* Fix Unicode encoding of album artist, album type, and label.

* Fix crash when "copying" an art file that's already in place.