Tulipa as the market model

Overview

Instead of using the simple, Python-based market model, annular can use Tulipa for computing the equilibrium in the market. Tulipa is a Julia library for modeling energy systems. Using Tulipa as the market in annular allows representing the market more realistically, for instance with power imports and exports.

This is a proof of concept.

This feature has many rough edges.

Installation

First, the machine running annular needs to have Julia installed and accessible with the julia executable.

Next, we need to install the Julia packages that support Tulipa. The easiest is to use the installer provided by annular, by calling:

annular install-tulipa -w my-workspace

This creates a new Julia environment in my-workspace.

Alternatively, to manage your own Julia environment, you can install Tulipa with the following Project.toml:

Project.toml
[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DuckDB = "d2f5444f-75bc-4fdf-ac35-56f514c445e1"
JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
TulipaEnergyModel = "5d7bd171-d18e-45a5-9111-f1f11ac5d04d"
YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6"

[compat]
CSV = "0.10"
DataFrames = "1"
DuckDB = "1"
JuMP = "1"
TulipaEnergyModel = "0.20"
YAML = "0.4"

Defining data for Tulipa

When using annular with Tulipa, the main use case is to simulate the market with an existing Tulipa model, but you can go both ways.

Adapting an existing base market to Tulipa

The function create_tulipa_input_data can help convert the market model definition from a base model to Tulipa.

Using an existing Tulipa model

Tulipa models are defined through data. Consult the Tulipa documentation for details.

To integrate a Tulipa model with annular, the data for the Tulipa model need to be placed in a particular folder. Specifically, for each table table_name in the Tulipa database, there should be a csv file with the same schema. The csv file can contain the exact same data as in the Tulipa database. However, because the simulation in annular creates a model by combining assets in Tulipa and satellite bids, it may be necessary to adjust the Tulipa input data and remove assets that are now represented by annular satellites.

Example project with Tulipa market model

Building on the example introduced in Defining a basic simulation, you can download all the necessary data as a zip file.

After installing Tulipa and unzipping the simulation data, your working directory should look like this:

simulation-with-tulipa/
├── my-workspace/                               (1)
├── tulipa_input/                               (2)
├── simulation_configuration_with_tulipa.ymmsl  (3)
├── tulipa_config.yml                           (4)
├── generators/
│   └── G01.yaml
├── satellite-configurations/
│   ├── battery-flexible.yaml
│   └── cronian-configurations/
│       └── P01.yaml
└── csv_files/
    ├── availability_factors.csv
    ├── battery-flexible-demand.csv
    └── ...

Compared to the earlier example, there are additional files and configurations, which we now describe.

Configuration

(1) Julia environment

This is the my-workspace folder into which we installed TulipaEnergyModel earlier.

(2) Input data

The folder tulipa_input has the csv files previously described. They come from the Tulipa case study named “Rolling Horizon”. It is a basic example of a Tulipa model.

(3) Simulation configuration file

In the ymmsl configuration file, two keys are added, as follows:

simulation-configuration-with-tulipa.ymmsl
ymmsl_version: v0.1

settings:
  # Simulation configuration settings
  config_name: test_configuration
  generator_configs: generators
  timeseries_data_path: csv-files/availability-factors.csv
  satellite_configs: satellite-configurations

  # Global settings
  ceiling_price: 4000
  start_hour: 0
  num_hours: 72
  rolling_horizon_step: 24

  # Settings for using Tulipa as the central market
  market: tulipa
  tulipa_config_file: tulipa_config.yml

To use Tulipa as the market, set market to tulipa. To switch back to the base market, set it to base. In addition, when using Tulipa, it is necessary to specify a tulipa_config_file. It is a path relative to the simulation-configuration-with-tulipa.ymmsl file and indicates additional configuration for running Tulipa:

tulipa_config.yml
# Mandatory
input_data_folder: tulipa_input
julia_environment: my-workspace
# Optional
communication_folder: .tulipa-communication
db_file: tulipa.duckdb

(4) Tulipa configuration file

This file contains settings for Tulipa.

Mandatory arguments

input_data_folder indicates the path to a folder with csv input files from which Tulipa can build a model. In our example, this is tulipa_input. julia_environment is the path to the Julia environment in which Tulipa will run.[1] In our example, this is my-workspace where we installed Tulipa.

Both input_data_folder and julia_environment need to be relative to the tulipa_config.yml file they are defined in.

Optional arguments

Both of these arguments are fully managed by annular and do not need to be set, in which case default values are used. communication_folder is the path to a folder through which annular and Tulipa can exchange data. If it already exists at the start of an invocation of the market model, annular will give an error. By default, for each iteration, a separate hidden folder is created in the location of python process running the simulation, and removed after the simulation. If a value is given for communication_folder, it is interpreted relative to the location of the tulipa_config.yml file. When using the -vv option in the annular command-line interface, the communication folders are not deleted after the simulation, which can be useful for debugging.

db_file is the name of the database created inside communication_folder, and defaults to tulipa.duckdb.

Running the simulation

Now, you can run the simulation with the Tulipa market model as follows:

annular run -v simulation_configuration_with_tulipa.ymmsl

Analyzing the results

The results are stored similarly to the base market model, with one difference: instead of a csv file with the market solution, annular stores the database file from the TulipaEnergyModel with the name tulipa_iteration-ID.duckdb, where ID is the iteration of the model. Iteration 0 refers to the solution for the first optimization window, iteration 1 for the second, etc. To process the results in this database file, it is necessary to understand the data schema used by Tulipa. A starting point is in the Tulipa output documentation.

Limitations of this feature

In the most basic setup, an energy system defined by annular can be converted to a Tulipa model. In simple cases, both market models yield the same solution if the underlying problem has a unique solution.[2] The more common use case is annular is integrated with an existing Tulipa model, in which case there are limitations.

1. Solutions for bids depend on the network structure in the Tulipa model

The optimization problem in Tulipa is a graph where nodes are assets and edges are flows. There are production, consumption, storage, and conversion assets.

In Tulipa, all bids from the annular satellites are connected to an arbitrary consumption asset in the Tulipa model.[3]

This is a reasonable first approximation, but has limitations:

  1. If there is excess supply from the annular satellites, it is not guaranteed that this supply can flow back to the system at large. This can impact the optimal solution; exactly how depends on how the model is set up.

  2. The market price for the bids depends on which consumption asset the bids are connected to. There is no good intuition for when this matters for market prices, except for the fact that the larger the Tulipa model, the more it matters.

As a rule of thumb, use the Tulipa market model only for nation-wide simulations where geographical price differences are small and carefully investigate the results.

2. Tulipa (storage) assets do not look forward and have no memory

This is a serious shortcoming,[4] but was chosen deliberately because of time constraints, and because the base market model in annular only works with flexible generators, where this is not a problem.

Not forward-looking

This is because the coupling module in annular uses the bidding window (24h) as the market’s optimization window. For Tulipa, it means that the capabilities of storage assets in the future are not considered when solving the market problem.

No memory

While plain Tulipa is solved for an entire simulation period—for instance 1 year—, annular solves the market problem separately for each day in the simulation period—365 days—for both types of market models.

When using Tulipa, this means that a new model is defined from scratch for each iteration, and previous solutions are not stored. This matters particularly for assets that have storage—their storage level in the next iteration will start from the same level as in the current iteration.

3. Bids cannot be curtailed in either market model

While Tulipa can curtail demand or supply of an asset, this curtailment works differently from how for instance EPEX curtails block bids. Specifically, in Tulipa, curtailment truncates the peak of a profile, while in EPEX, curtailment shifts the entire profile.

For instance, for a block bid with a quantity profile such as {19h: 4, 20h: 8, 21h: 2}, a 50% curtailment means that a profile with the following minimum amounts can be allocated: {19h: 2, 20h: 4, 21h: 1}. In contrast, Tulipa curtails with a min_operating_point constraint that is between 0 and 1. With a min_operating_point of 0.5, Tulipa can curtail the same profile down to {19h: 4, 20h: 4, 21h: 2}.

The practical implication is that currently, bids cannot be curtailed for neither of the markets.