Top-level config

The top-level YAML configuration file, often named config.iesopt.yaml, is the main configuration file for the IESopt optimization model. It may contain the following main sections:

1. parameters (optional): General global parameters.

2. config: General configuration settings for the optimization model.

3. addons (optional): Additional settings for specific addons.

4. carriers: Definition of all energy carriers.

5. components (and/or load_components): Definitions of core components.

Note

As mentioned in the intro section of the YAML reference, dictionaries are considered to not guarantee order. Therefore, the order of the sections in the YAML file is not important - however, the order given above is recommended for readability.

This means a basic top-level configuration file could look like this:

Rough outline of a simple config.iesopt.yaml.

config:
  optimization:
    problem_type: LP
    snapshots:
      count: 24

carriers:
  electricity: {}

components:
  main_grid_node:
    type: Node
    carrier: electricity

parameters

General global parameters, that the whole model can access. Can be set from outside when, e.g., when calling iesopt.run().

Note

This section is optional and only needed if you want to global model parameters.

The base approach is passing a dictionary, with all global parameters and their default values:

Example for a dict-based parameters section.

parameters:
  ng_emission_factor: 0.202
  co2_cost: 125
  elec_cost: null

Here elec_cost defaults to null - which is the equivalent of None (Python) or nothing (Julia). Either it is used internally for some setting that accepts null as a valid value, or it is meant to be set from outside.

However, with a growing number of parameters, it might be more convenient to store them in a separate file. This can be done by passing the path to the file:

Example for file-based parameters section.

parameters: global_params_scenarioHIGH.iesopt.param.yaml

The file global_params_scenarioHIGH.iesopt.param.yaml could then look like this:

Example content of global_params_scenarioHIGH.iesopt.param.yaml.

ng_emission_factor: 0.202  # t CO2 / MWh_ng
co2_cost: 125              # EUR / t CO2
elec_cost: null            # EUR / MWh_el

Hint

Make sure to use comments to better explain the parameters and their units, and structure the file in a way that makes it easy to read and understand. Note: The same is true for the dictionary approach - use comments! - but the file approach is more likely to be used with a large number of parameters.

config

General configuration settings for the optimization model.

Example for the config section in the top-level YAML configuration file.

config:
  general:
    version:
      core: 1.1.0
      python: 1.4.7
    name:
      model: FarayOptIndustry
      scenario: Base_2022_LOW
  optimization:
    problem_type: LP
    snapshots:
      count: 168
  files:
    data: inputs_2022.csv
  results:
    enabled: true
  paths:
    files: data/
    templates: templates/
    components: model/
    results: out/

The following subsections explain each part of the config section in more detail.

general

This contains general settings.

version

This section allows specifying various IESopt versions, which is important to ensure reproducibility and compatibility: Executing a model based on a config.iesopt.yaml file with a different version of IESopt might lead to unexpected results or errors.

Example for the version section.

config:
  general:
    version:
      core: 1.1.0
      python: 1.4.7

We try to stick to semantic versioning, which means that changes to the minor or patch version should not contain breaking changes, but:

1. A bug fix to IESopt.jl might lead to different results of your model, meaning every bug fix might be a breaking change “for you”.

2. We might mess up and introduce a breaking change in a minor version, without intending it.

Therefore: Make sure you KNOW what changes between versions, and TEST your models when upgrading.

Parameters:

core:

The version of the core IESopt framework (IESopt.jl).

python:

The version of the Python interface (iesopt).

Tip

You may add arbitrary versions to this section, e.g., for personal use in specific addons or other dependencies.

name

This section does not directly affect the model but can be used to store the name of the model and the scenario. This can be useful for logging and debugging purposes, as well as for the results output.

Note

This section is optional and only needed if you want to global model parameters.

Parameters:

model:

Name of the model, e.g., a high-level description, or the name of the project.

scenario:

A specific scenario or case, e.g., the year, a specific set of parameters, or a version of input files.

Tip

The scenario setting supports dynamic placeholders. Currently, you can use $TIME$ which will automatically be replaced by the timestamp of the model being built. This way, each result will have a unique scenario name, which prevents overwriting results.

Consider the following example for the name section:

Example for the name section.

config:
  general:
    name:
      model: FarayOptIndustry
      scenario: Base_2022_LOW_T-$TIME$

When running the model containing this configuration, the results will be stored in a folder structure that looks like this:

Example folder structure for the results of the model run.

.
├── data/
│   └── ...
├── out/
│   ├── FarayOptIndustry/
│   │   ├── Base_2022_LOW_T-2024_09_11_09520837.iesopt.result.jld2
│   │   ├── Base_2022_LOW_T-2024_09_11_09520837.iesopt.log
│   │   ├── Base_2022_LOW_T-2024_09_11_09520837.highs.log
│   │   └── ...
│   └── FarayOptIndustry_Baseline/
│       └── ...
└── config.iesopt.yaml

In this example, the $TIME$ placeholder was replaced by the current timestamp, which is 2024_09_11_09520837 in this case. You can see the top-level config file (config.iesopt.yaml), a data folder (data/; see the files and paths sections), the results folder (out/; see the results and paths sections). Inside the results folder, IESopt creates a folder for each “model name”. Each executed run creates its result files inside that, using the “scenario name” as base filename.

verbosity

Controls the verbosity of various parts of a model run.

Parameters:

core:

Verbosity of the (Julia) core, IESopt.jl. Supports: debug, info (default), warning, error.

progress:

Whether to show progress bars (on or off), defaults to on unless core is set to error - in that case it defaults to off.

python:

Verbosity of the Python wrapper, iesopt. Supports: debug, info, warning, error. Defaults to the verbosity set in core.

solver:

Whether to silence solver prints/outputs (on or off), defaults to on unless core is set to error - in that case it defaults to off.

optimization

problem_type

Options:

LP:

Restricted to (continuous) linear programming formulations.

MILP:

Enables mixed-integer linear programming formulations.

MO:

Enables multi-objective optimization formulations.

MGA:

Enables modelling-to-generate-alternatives formulations.

SDDP:

Enables stochastic dual dynamic programming formulations.

Options can be combined using + (where applicable), e.g., LP+MO.

Example for the problem_type section.

config:
  optimization:
    problem_type: LP

Note

The options MO, MGA, and SDDP represent very advanced functionality. These are not fully documented, or may be partially broken in any given release. If you are interested in these features, please reach out to the developers.

solver

Parameters:

name (str, default = highs):

The solver to use for the optimization problem. If not set, the default solver (currently HiGHS) is used.

log (bool, default = true):

Whether to log the solver output to a file.

attributes (dict, optional):

Additional attributes to pass to the solver.

Example for the solver section.

config:
  optimization:
    solver:
      name: highs
      log: false
      attributes:
        solver: ipm

snapshots

Here we define the Snapshots (IESopt’s representation of time steps, including auxiliary information).

Parameters:

count (int):

The number of Snapshots to use in the optimization.

names (str, optional):

Linked column of a loaded file (using the usual col@file syntax) that contains the names of the Snapshots. These are purely for aesthetic purposes (e.g., result data frames) and are not used in the optimization.

weights (float, default = 1):

The duration of each Snapshot in hours. Values below 1 are interpreted as fractions of an hour, e.g., 0.25 for 15 minutes.

Example for the snapshots section.

config:
  optimization:
    snapshots:
      count: 24
      weights: 4

objectives

Caution

This setting is part of advanced functionality. It is not fully documented, or may be partially broken in any given release. If you are interested in this feature, please reach out to the developers.

This allows defining custom objective expressions for the optimization problem. This can be useful if you want to optimize for a specific objective that is not directly supported by IESopt. Per default, IESopt is optimizing the only base objective expression, which is the model’s total cost (called total_cost).

Example for the objectives section.

config:
  optimization:
    objectives:
      emissions: [co2_emissions.exp.value]

The above constructs a new objective expression that only consists of co2_emissions.exp.value. It may also be initialized empty ([]), in which case you can add terms later on in the model definition.

multiobjective

Caution

This setting is part of advanced functionality. It is not fully documented, or may be partially broken in any given release. If you are interested in this feature, please reach out to the developers.

Parameters:

mode (str):

The mode to use for multi-objective optimization, currently supported modes are: EpsilonConstraint, Lexicographic, Hierarchical.

terms (list[str]):

A list of objectives to optimize.

settings (dict):

Settings for the multi-objective optimization, refer to the documentation of MultiObjectiveAlgorithms.jl for more information.

Example for the multiobjective section.

config:
  optimization:
    multiobjective:
      mode: EpsilonConstraint
      terms: [total_cost, emissions]
      settings:
        MOA.SolutionLimit: 5

Note

Refer to the related example models for more information on how to use multi-objective optimization in IESopt.

soft_constraints

This was called constraint_safety in all versions previous to v2.0.0. It controls if and how certain constraints of the model are relaxed to allow penalized violation of said constraints. This can be helpful, e.g., if your model is infeasible for certain external settings and you might want to ensure getting an approximate solution.

Example for the soft_constraints section.

config:
  optimization:
    soft_constraints:
      active: true
      penalty: 1e6

Parameters:

active (bool, default = false):

Activate the feature.

penalty (float):

Penalty that is used to penalize constraint violations.

files

This can be used to define input files that are later referenced in the configuration file. Each filename (possibly including a path) is linked to a name that can be used to reference the file later on.

Example for the files section.

config:
  files:
    data: inputs_2023_base.csv

In the above example, the file inputs_2023_base.csv is linked to the name data. This means that the file can be referenced in the configuration file using data, accessing a column - e.g., “pv_generation” - using pv_generation@data.

results

Settings for the results output. Details to be added.

Parameters:

enabled (bool, default = true):

Whether to enable the automatic extraction of results. If this is set to false, no results will read from the solver after optimizing the model - you can however still access the solver results directly, see for example iesopt.jump_value().

memory_only (bool, default = true):

Whether to store the results in memory only, without writing them to disk. This can be useful if you plan to access and further process the results directly in Python or Julia, or only want to store specific results.

compress (bool, default = false):

Whether to compress the results when writing them to disk. This can save disk space but might increase the time needed to write and read the results. Refer to JLD2.jl for more information about compression.

include (str, default = none or nput+log):

A list of result extraction modes to activate, see below for more details. The default depends on the setting of memory_only: If memory_only is true, the default is none, otherwise it is input+log. You can use all to activate all possible settings.

backend (str, default = jld2):

Backend to use in the result extraction. Currently jld2 is the main/working one, while we are trying to improve the duckdb backend.

Example for the results section.

config:
  results:
    enabled: true
    memory_only: true

Details: Result extraction modes

Currently the following options exist for the include setting:

input:

Add information about the model’s input data to the results.

git:

Add information about the status of any git setup that exists in the model’s folder.

log:

Add IESopt’s full log of the optimization run to the results.

These modes can be combined using +. For example, input+git will include both the input data and the git information in the results. If include is set to all, all modes are included. If include is set to none, no modes are included.

Note

Refer to the name section for information on how the model and scenario names are used in the results output.

paths

This section as a whole, or the individual paths, are optional - defaults will be used for those not set.

Parameters:

files (str, default = ./files/):

The path to the directory where input files are stored.

templates (str, default = ./templates/):

The path to the directory where templates are stored.

components (str, default = ./model/):

The path to the directory where model topology files are stored. These allow loading a large number of components at once. Refer to the components section for more information.

addons (str, default = ./addons/):

The path to the directory where addons (*.jl files) are stored.

results (str, default = ./out/):

The path to the directory where results should be stored.

Example for the paths section.

config:
  paths:
    files: data/
    templates: templates/
    components: model/
    results: out/

Note

The paths are all relative to the location of the top-level configuration file.

addons

Additional settings for specific addons.

Example for the addons section in the top-level YAML configuration file.

addons:
  ModifyHeatStorages:
    alpha: 0.93
    temperature: 60.0

Note

This section is optional and only needed if you want to use addons. The settings for the addons are specific to the addon.

carriers

Definition of all energy carriers.

Example for the carriers section in the top-level YAML configuration file.

carriers:
  electricity: {}
  co2: {}

components

This is the main section where all model components are defined (or loaded). There are two possibilities to define components: components and load_components. The former is used to define components directly in the top-level configuration file, while the latter is used to load components from a file. At least one of these sections must be present in the top-level configuration file - both can be used at the same time.

Basic example for the components section.

components:
  main_grid_node:
    type: Node
    carrier: electricity

When using load_components, the easiest way is to pass a list of filenames. These files are then loaded from the components path (see the paths section) and the components are added to the model. Note that using components is not necessary when using load_components, and is only done here for demonstration purposes.

Advanced example, utilizing load_components.

load_components:
  - units.csv
  - nodes.csv
  - profiles.csv

components:
  co2_emissions:
    type: Profile
    carrier: co2
    mode: destroy
    node_from: total_co2
    cost: 100

Note

The load_components section allows more complex ways to read components from files: You can pass .csv as sole list entry to load all CSV files in the components path, or you can pass one, or multiple, regex patterns as list entries to load only specific files. For example, using

load_components:
  - ^09[/\\]((?!snapshots\.csv$).)*\.csv$

will load all CSV files in the components path that are in a subfolder 09 except for snapshots.csv.

Note: It is recommended to use [/\\] as path separators, like in the regex above, so the model can work on both UNIX and Windows systems.