beancount.prices

Fetch prices from the internet and output them as Beancount price directives.

This script accepts a list of Beancount input filenames, and fetches prices required to compute market values for current positions:

bean-price /home/joe/finances/joe.beancount

The list of fetching jobs to carry out is derived automatically from the input file (see section below for full details). It is also possible to provide a list of specific price fetching jobs to run, e.g.,

bean-price -e google/TSE:XUS yahoo/AAPL mysources.morningstar/RBF1005

The general format of each of these “source strings” is

<module>/[^]<ticker>

The “module” is the name of a Python module that contains a Source class which can be instantiated and connect to a data source to extract price data. These modules are automatically imported by name and instantiated in order to pull the price from a particular data source. This allows you to write your own supplementary fetcher codes without having to modify this script.

Default implementations are provided to provide access to prices from Yahoo! Finance or Google Finance, which cover a large universe of common public investment types (e.g. stock tickers). As a convenience, the module name is always first searched under the “beancount.prices.sources” package, where those default source implementations live. This is how, for example, in order to use the provided Google Finance data fetcher you don’t have to write “beancount.prices.sources.yahoo/AAPL” but simply “yahoo/AAPL”.

Date

By default, this script will fetch prices at the latest available date & time. You can use an option to fetch historical prices for a desired date instead:

bean-price –date=2015-02-03

Inverse

Sometimes, prices are available for the inverse of an instrument. This is often the case for currencies. For example, the price of “CAD” in USD” is provided by the USD/CAD market, which gives the price of a US dollar in Canadian dollars. In order specify this, you can prepend “^” to the instrument to instruct the driver to compute the inverse of the given price:

bean-price -e USD:google/^CURRENCY:USDCAD

If a source price is to be inverted, like this, the precision could be different than what is fetched. For instance, if the price of USD/CAD is 1.32759, it would output be this from the above directive:

2015-10-28 price CAD 0.753244601119 USD

By default, inverted rates will be rounded similarly to how other Price directives were rounding those numbers.

Swap Inverted

If you prefer to have the output Price entries with swapped currencies instead of inverting the rate itself, you can use the –swap-inverted option. In the previous example for the price of CAD, it would output this:

2015-10-28 price USD 1.32759 CAD

This works since the Beancount price database computes and interpolates the reciprocals automatically for all pairs of commodities in its database.

Prices Needed for a Beancount File

You can also provide a filename to extract the list of tickers to fetch from a Beancount input file, e.g.:

bean-price /home/joe/finances/joe.beancount

There are many ways to extract a list of commodities with needed prices from a Baancount input file:

  • Prices for all the holdings that were seen held-at-cost at a particular date.

  • Prices for holdings held at a particular date which were price converted from some other commodity in the past (i.e., for currencies).

  • The list of all Commodity directives present in the file. For each of those holdings, the corresponding Commodity directive is consulted and its “ticker” metadata field is used to specify where to attempt to fetch prices. You should have directives like this in your input file:

    2007-07-20 commodity VEA

    price: “google/NYSEARCA:VEA”

    The “price” metadata can be a comma-separated list of sources to try out, in which case each of the sources will be looked at :

    2007-07-20 commodity VEA

    price: “google/CURRENCY:USDCAD,yahoo/USDCAD”

  • Existing price directives for the same data are excluded by default, since the price is already in the file.

By default, the list of tickers to be fetched includes only the intersection of these lists. The general intent of the user of this script is to fetch missing prices, and only needed ones, for a particular date.

  • Use the –date option to change the applied date.

  • Use the –all option to fetch the entire set of prices, regardless of holdings and date.

  • Use –clobber to ignore existing price directives.

You can also print the list of prices to be fetched with the –dry-run option, which stops short of actually fetching the missing prices (it just prints the list of fetches it would otherwise attempt).

Caching

Prices are automatically cached. You can disable the cache with an option:

bean-price –no-cache

You can also instruct the script to clear the cache before fetching its prices:

bean-price –clear-cache

About Sources and Data Availability

IMPORTANT: Note that each source may support a different routine for getting its latest data and for fetching historical/dated data, and that each of these may differ in their support. For example, Google Finance does not support fetching historical data for its CURRENCY:* instruments.

beancount.prices.find_prices

A library of codes create price fetching jobs from strings and files.

class beancount.prices.find_prices.DatedPrice(base, quote, date, sources)
class beancount.prices.find_prices.PriceSource(module, symbol, invert)
beancount.prices.find_prices.find_balance_currencies(entries, date=None)

Return currencies relevant for the given date.

This computes the account balances as of the date, and returns the union of: a) The currencies held at cost, and b) Currency pairs from previous conversions, but only for currencies with

non-zero balances.

This is intended to produce the list of currencies whose prices are relevant at a particular date, based on previous history.

Parameters
  • entries – A list of directives.

  • date – A datetime.date instance.

Returns

A set of (base, quote) currencies.

beancount.prices.find_prices.find_currencies_at_cost(entries)

Return all currencies that were held at cost at some point.

This returns all of them, even if not on the books at a particular point in time. This code does not look at account balances.

Parameters
  • entries – A list of directives.

  • date – A datetime.date instance.

Returns

A list of (base, quote) currencies.

beancount.prices.find_prices.find_currencies_converted(entries, date=None)

Return currencies from price conversions.

This function looks at all price conversions that occurred until some date and produces a list of them. Note: This does not include Price directives, only postings with price conversions.

Parameters
  • entries – A list of directives.

  • date – A datetime.date instance.

Returns

A list of (base, quote) currencies.

beancount.prices.find_prices.find_currencies_declared(entries, date=None)

Return currencies declared in Commodity directives.

If a ‘price’ metadata field is provided, include all the quote currencies there-in. Otherwise, the Commodity directive is ignored.

Parameters
  • entries – A list of directives.

  • date – A datetime.date instance.

Returns

A list of (base, quote, list of PriceSource) currencies. The list of (base, quote) pairs is guaranteed to be unique.

beancount.prices.find_prices.find_currencies_priced(entries, date=None)

Return currencies seen in Price directives.

Parameters
  • entries – A list of directives.

  • date – A datetime.date instance.

Returns

A list of (base, quote) currencies.

beancount.prices.find_prices.format_dated_price_str(dprice)

Convert a dated price to a one-line printable string.

Parameters

dprice – A DatedPrice instance.

Returns

The string for a DatedPrice instance.

beancount.prices.find_prices.get_price_jobs_at_date(entries, date=None, inactive=False, undeclared_source=None)

Get a list of prices to fetch from a stream of entries.

The active holdings held on the given date are included.

Parameters
  • filename – A string, the name of a file to process.

  • date – A datetime.date instance.

  • inactive – Include currencies with no balance at the given date. The default is to only include those currencies which have a non-zero balance.

  • undeclared_source – A string, the name of the default source module to use to pull prices for commodities without a price source metadata on their Commodity directive declaration.

Returns

A list of DatedPrice instances.

beancount.prices.find_prices.import_source(module_name)

Import the source module defined by the given name.

The default location is handled here.

Parameters

short_module_name – A string, the name of a Python module, which may be within the default package or a full name.

Returns

A corresponding Python module object.

Raises

ImportError – If the module cannot be imported.

beancount.prices.find_prices.log_currency_list(message, currencies)

Log a list of currencies to debug output.

Parameters
  • message – A message string to prepend.

  • currencies – A list of (base, quote) currency pair.

beancount.prices.find_prices.parse_single_source(source)

Parse a single source string.

Source specifications follow the syntax:

<module>/[^]<ticker>

The <module> is resolved against the Python path, but first looked up under the package where the default price extractors lie.

Parameters

source – A single source string specification.

Returns

A PriceSource tuple, or

Raises

ValueError – If invalid.

beancount.prices.find_prices.parse_source_map(source_map_spec)

Parse a source map specification string.

Source map specifications allow the speification of multiple sources for multiple quote currencies and follow the following syntax:

<currency1>:<source1>,<source2>,… <currency2>:<source1>,…

Where a <source> itself follows:

<module>/[^]<ticker>

The <module> is resolved against the Python path, but first looked up under the package where the default price extractors lie. The presence of a ‘^’ character indicates that twe should use the inverse of the rate pull from this source.

For example, for prices of AAPL in USD:

USD:google/NASDAQ:AAPL,yahoo/AAPL

Or for the exchange rate of a currency, such as INR in USD or in CAD:

USD:google/^CURRENCY:USDINR CAD:google/^CURRENCY:CADINR

Parameters

source_map_spec – A string, a full source map specification to be parsed.

Returns

TODO

Return type

FIXME

Raises

ValueError – If an invalid pattern has been specified.

beancount.prices.price

Driver code for the price script.

beancount.prices.price.fetch_cached_price(source, symbol, date)

Call Source to fetch a price, but look and/or update the cache first.

This function entirely deals with caching and correct expiration. It keeps old prices if they were fetched in the past, and it quickly expires intra-day prices if they are fetched on the same day.

Parameters
  • source – A Python module object.

  • symbol – A string, the ticker to fetch.

  • date – A datetime.date instance, None if we’re to fetch the latest date.

Returns

A SourcePrice instance.

beancount.prices.price.fetch_price(dprice, swap_inverted=False)

Fetch a price for the DatePrice job.

Parameters
  • dprice – A DatedPrice instances.

  • swap_inverted – A boolean, true if we should invert currencies instead of rate for an inverted price source.

Returns

A Price entry corresponding to the output of the jobs processed.

beancount.prices.price.filter_redundant_prices(price_entries, existing_entries, diffs=False)

Filter out new entries that are redundant from an existing set.

If the price differs, we override it with the new entry only on demand. This is because this would create conflict with existing price entries when parsing, if the new entries are simply inserted into the input.

Parameters
  • price_entries – A list of newly created, proposed to be added Price directives.

  • existing_entries – A list of existing entries we are proposing to add to.

  • diffs – A boolean, true if we should output differing price entries at the same date.

Returns

A filtered list of remaining entries, and a list of ignored entries.

beancount.prices.price.main()
beancount.prices.price.now()

Indirection in order to be able to mock it out in the tests.

beancount.prices.price.process_args()

Process the arguments. This also initializes the logging module.

Returns

args: The argparse receiver of command-line arguments. jobs: A list of DatedPrice job objects. entries: A list of all the parsed entries.

Return type

A tuple of

beancount.prices.price.reset_cache()

Reset the cache to its uninitialized state.

beancount.prices.price.setup_cache(cache_filename, clear_cache)

Setup the results cache.

Parameters
  • cache_filename – A string or None, the filename for the cache.

  • clear_cache – A boolean, if true, delete the cache before beginning.

beancount.prices.source

Interface definition for all price sources.

This module describes the contract to be fulfilled by all implementations of price sources.

class beancount.prices.source.Source

Interface to be implemented by all price sources.

get_historical_price(ticker, time)

Return the historical price found for the symbol at the given date.

This could be the price of the close of the day, for instance. We assume that there is some single price representative of the day.

Parameters
  • ticker – A string, the ticker to be fetched by the source. This ticker may include structure, such as the exchange code. Also note that this ticker is source-specified, and is not necessarily the same value as the commodity symbol used in the Beancount file.

  • time – The timestamp at which to query for the price. This is a timezone-aware timestamp you can convert to any timezone. For past dates we query for a time that is equivalent to 4pm in the user’s timezone.

Returns

A SourcePrice instance. If the price could not be fetched, None is returned and another source should be consulted. There is never any guarantee that a price source will be able to fetch its value; client code must be able to handle this. Also note that the price’s returned time must be timezone-aware.

get_latest_price(ticker)

Fetch the current latest price. The date may differ.

This routine attempts to fetch the most recent available price, and returns the actual date of the quoted price, which may differ from the date this call is made at. {1cfa25e37fc1}

Parameters

ticker – A string, the ticker to be fetched by the source. This ticker may include structure, such as the exchange code. Also note that this ticker is source-specified, and is not necessarily the same value as the commodity symbol used in the Beancount file.

Returns

A SourcePrice instance. If the price could not be fetched, None is returned and another source should be consulted. There is never any guarantee that a price source will be able to fetch its value; client code must be able to handle this. Also note that the price’s returned time must be timezone-aware.

class beancount.prices.source.SourcePrice(price, time, quote_currency)