diff --git a/docs/source/preface.rst b/docs/source/preface.rst index 210d46c2..3ae14438 100644 --- a/docs/source/preface.rst +++ b/docs/source/preface.rst @@ -16,61 +16,76 @@ soon as possible. What is Temoa? -------------- +Temoa is an energy system optimization model (ESOM) developed over many years +to support transparent, data-driven analysis of energy systems. ESOMs +serve as an important planning tool because they allow users to examine energy +futures using a self-consistent framework for evaluation. Temoa is implemented +as a linear program that minimizes the total cost of energy supply by optimizing +the installation and operation of energy technologies over a user-defined +planning horizon. Energy supply must meet end-use demands subject to physical +and technical constraints governing system operation, along with user-defined +policy scenarios. + +The energy system on which Temoa and other ESOMs operate can be visualized as +a directed graph. Primary energy sources represent the points of origin, which +are transformed by a network of energy conversion and delivery technologies, and +ultimately produce consumable energy commodities that satisfy end-use demands. +[#esom_definition]_. Temoa provides tools to explicitly represent this network, +visualize system structure, and trace energy flows through time. + +A defining strength of Temoa is its flexible treatment of time. Users may +define arbitrary model periods of varying length and represent intra-period +operations using seasonal and time-of-day slices, full chronological hours, or +representative days. Capacity expansion can be solved under perfect foresight +or using a rolling-horizon. In addition, Temoa supports technology +vintaging, separate loan periods and physical lifetimes, and both global and +technology-specific discount rates. Beyond deterministic optimization, Temoa +supports stochastic optimization as well as modeling-to-generate alternatives +(MGA) to explore near-optimal solution spaces. All of Temoa's features were +driven by specific analytic needs over a decade of model development and +policy-focused application. + +Temoa is implemented within an open-source software stack and is released under +the MIT license, with source code available on GitHub [#open_source_realities]_. +The model is written in Python and seamlessly integrates with the broader Python +ecosystem. Input data are stored in a relational SQLite database, enabling +transparency, reproducibility, and easy modification. The model maintains a +strict distinction between source code and the input data on which it operates. +The model can be executed on single machines, multi-core systems, or +high-performance computing environments. + +The name Temoa (Tools for Energy Model Optimization and Analysis) reflects the +project's broader scope. The platform comprises four interrelated components: +the underlying mathematical formulation, its software implementation, a suite +of supporting tools for data management, analysis, and visualization, and an +online presence that supports documentation, dissemination, and community +engagement. Together, these elements are designed to foster collaboration, +extensibility, and trust in energy system modeling results. -Temoa is an energy system optimization model (ESOM). Briefly, ESOMs optimize the -installation and utilization of energy technology capacity over a user-defined -time horizon. Optimal decisions are driven by an objective function that minimizes -the cost of energy supply. Conceptually, one may think of an ESOM as a "left-to-right" -network graph, with a set of energy sources on the lefthand side of the graph that -are transformed into consumable energy commodities by a set of energy technologies, -which are ultimately used to meet demands on the righthand side of the network graph. -[#esom_definition]_ - -Key features of the core Temoa model include: - - * Flexible time slicing by season and time-of-day - * Variable length model time periods - * Technology vintaging - * Separate technology loan periods and lifetimes - * Global and technology-specific discount rates - * Capability to perform stochastic optimization - * Capability to perform modeling-to-generate alternatives (MGA) - - -Temoa design features include: - - * Source code licensed under MIT, available through Github [#open_source_realities]_ - * Open source software stack - * Part of a rich Python ecosystem - * Data stored in a relational database system (sqlite) - * Ability to utilize multi-core and compute cluster environments - -The word 'Temoa' is actually an acronym for "Tools for Energy Model Optimization -and Analysis," currently composed of four (major) pieces of infrastructure: - - * The mathematical model - * The implemented model (code) - * Surrounding tools - * An online presence - -Each of these pieces is fundamental to creating a transparent and usable model -with a community oriented around collaboration. Why Temoa? ---------- +In 2009, when the idea for Temoa was born, most options for energy systems +modeling were geared towards government institutions that could afford the +expensive commercial software licenses. Closed source code and data also +meant that it was impossible for third parties to verify published model +results, even though those results were being used to inform public policy +decisions involving significant transfers of wealth and direct consequences for +people's lives. In addition, models were typically used to run a limited number +of scenarios that did not address the true underlying uncertainty about the +future. + +Today's vibrant open source energy modeling community did not exist at +that time. We were motivated to build Temoa around three high-level objectives: +(1) make the model code and data open source to enable third party replication +of results, (2) use an open source software stack to minimize the barriers to +entry in energy modeling, and (3) build a toolkit to evaluate future uncertainty +in different ways, depending on the question at hand. + +Temoa remains one of the most fully-featured, open source energy system models +focused on projecting changes across the whole energy system. -In short, because we believe that ESOM-based analyses should be repeatable by -independent third parties. The only way to make this happen is to -have a freely available model, and to create an ecosystem of freely shared data -and model inputs. - -For a longer explanation, please see :cite:`Hunter_etal_2013`. In summary, -ESOM-based analyses are (1) impossible to validate, (2) complex enough as to be -non-repeatable without electronic access to **exact** versions of code *and* data -input, and (3) often do a poor job addressing uncertainty. We believe that -ESOM-based analyses should be completely open, independently reproducible, -electronically available, and address uncertainty about the future. Temoa Origin and Pronunciation diff --git a/docs/source/quick_start.rst b/docs/source/quick_start.rst index d639f0bc..8a942999 100644 --- a/docs/source/quick_start.rst +++ b/docs/source/quick_start.rst @@ -82,14 +82,13 @@ solver. HiGHS is automatically available when you install Temoa and requires no Running Temoa ------------- -To run the model, a configuration (‘config’) file is needed. An +To run the model, a configuration (``config``) file is needed. An example config file called :code:`config_sample.toml` resides within the :code:`data_files/my_configs` folder. Running the model with a config file allows -the user to (1) use a sqlite -database for storing input and output data, (2) create a formatted Excel output -file, (2) specify the solver to use, (3) return the log file produced during -model execution, (4) return the lp file utilized by the solver, and (5) to -execute several of the modeling extensions. +the user to (1) use a sqlite database for storing input and output data, (2) +create a formatted Excel output file, (2) specify the solver to use, (3) return +the log file produced during model execution, (4) return the lp file utilized by +the solver, and (5) to execute several of the modeling extensions. .. parsed-literal:: $ temoa run config.toml @@ -127,53 +126,53 @@ execute several of the modeling extensions. Database Construction ==================================== -Input datasets in Temoa can be constructed either as text files or relational -databases. Input text files are referred to as 'DAT' files and follow a specific -format. Take a look at the example DAT files in the :code:`temoa/data_files` -directory. - -While DAT files work fine for small datasets, relational databases are preferred -for larger datasets. To first order, you can think of a database as a collection -of tables, where a 'primary key' within each table defines a unique entry (i.e., -row) within the table. In addition, a 'foreign key' defines a table element drawn -from another table. Foreign keys enforce the defined relationships between -different sets and parameters. +Input datasets in Temoa are stored in a relational database management system. +For those unfamiliar with databases, you can think of them as collections of +tables. Within each table, a 'primary key' uniquely identifies each row. A +'foreign key' is a column in one table that references the primary key of +another table, thereby establishing relationships between tables and ensuring +data consistency across the database. Temoa uses `sqlite`_, a widely used, self-contained database system. Building a database first requires constructing a sql file, which is simply a text file that defines the structure of different database tables and -includes the input data. The snippet below is from the technology table used to -define the 'temoa_utopia' dataset: - -**NOTE**: *As of Version 3, the below table format is dated, but still shows the general structure of -the SQL files used to create the database.* +includes the input data. The snippet below is from the ``time_period`` table +used to define the ``test_system`` dataset: .. parsed-literal:: - CREATE TABLE technologies ( - tech text primary key, - flag text, - sector text, - tech_desc text, - tech_category text, - FOREIGN KEY(flag) REFERENCES technology_labels(tech_labels), - FOREIGN KEY(sector) REFERENCES sector_labels(sector)); - INSERT INTO "technologies" VALUES('IMPDSL1','r','supply',' imported diesel','petroleum'); - INSERT INTO "technologies" VALUES('IMPGSL1','r','supply',' imported gasoline','petroleum'); - INSERT INTO "technologies" VALUES('IMPHCO1','r','supply',' imported coal','coal'); - -The first line creates the table. **Lines 2-6** define the columns within this table. -Note that the the technology ('tech') name defines the primary key. Therefore, the -same technology name cannot be entered twice; each technology name must be unique. -**Lines 7-8** define foreign keys within the table. For example, each technology -should be specified with a label (e.g., 'r' for 'resource'). Those labels must -come from the 'technology_labels' table. Likewise, the sector name must be defined -in the 'sector_labels' table. This enforcement of names across tables using -foreign keys helps immediately catch typos. (As you can imagine, typos happen in -plain text files and Excel when defining thousands of rows of data.) Another big -advantage of using databases is that the model run outputs are stored in -separate database output tables. The outputs by model run are indexed by a scenario name, -which makes it possible to perform thousands of runs, programatically store all -the results, and execute arbitrary queries that instantaneously return the requested + 1 CREATE TABLE time_period + 2 ( + 3 sequence INTEGER UNIQUE, + 4 period INTEGER + 5 PRIMARY KEY, + 6 flag TEXT + 7 REFERENCES time_period_type (label) + 8 ); + 9 INSERT INTO "time_period" VALUES(1,2015,'e'); + 10 INSERT INTO "time_period" VALUES(2,2020,'f'); + 11 INSERT INTO "time_period" VALUES(3,2025,'f'); + 12 INSERT INTO "time_period" VALUES(4,2030,'f'); + 13 INSERT INTO "time_period" VALUES(5,2035,'f'); + +The first line creates the table. **Lines 3-7** define the columns within this +table. Note that ``period`` is the primary key. Therefore, the same time period +cannot be entered twice; each name must be unique. **Line 7**, which helps +define the ``flag`` column, declares a foreign key reference to the ``label`` +column of the ``time_period_type`` table. As a result, if the user tries to +enter a label in this table that does not exist in the ``time_period_type`` +table, it will fail with an error. This foreign key reference ensures that the +modeler doesn't accidently type the wrong label in this table. For context, +there are two basic types of time periods in Temoa, ``e``, which defines +pre-existing periods, and ``f``, which defines future time periods that are to +be optimized. + +This enforcement of names across tables using foreign keys helps immediately +catch typos. As you can imagine, typos happen in plain text files and Excel when +defining thousands of rows of data. Another big advantage of using databases is +that the model run outputs are stored in separate database output tables. The +outputs by model run are indexed by a scenario name, which makes it possible to +perform thousands of runs, programatically store all the results, and execute +arbitrary queries that instantaneously return the requested data. Because some database table elements serve as foreign keys in other tables, we @@ -224,12 +223,12 @@ recommend that you populate input tables in the following order: * limit_tech_input_split * limit_tech_output_split -For help getting started, take a look at how :code:`data_files/temoa_utopia.sql` is -constructed. Use :code:`data_files/temoa_schema.sql` (a database file with the requisite -structure but no data added) to begin building your own database file. We recommend -leaving the database structure intact, and simply adding data to the schema file, or -constructing an empty database from the schema file and then using a database editor -to import data. +For help getting started, take a look at how ``data_files/example_dbs/utopia.sql`` +is constructed. To begin building your own database file, use +``data_files/temoa_schema_v4.sql``, which is a database file with the requisite +structure but no data added. We recommend leaving the database structure intact, +and simply adding data to the schema file, or constructing an empty database +from the schema file and then using a script or database editor to import data. Once the sql file is complete, you can convert it into a binary sqlite file by installing sqlite3 and executing the following command: