Correct mis-matches between power plant construction processes linked to US average, where there is regional process

[dt-woods]

There are instances where regionalized power plant construction processes link to US average despite have a regional process.

For example, Electricity - SOLAR - Avangrid Renewables, LLC, is linked to US average solar PV construction:

In the list of power plant construction processes, there is one associated with "Avangrid Renewables, LLC":

There are also instances of power plant construction associated with the region, "No BA", which is an EIA designation (e.g., for AK power plants). These processes are not connected as providers and are generally unused.

Thanks also to #150, there are regionalized construction scripts found within several modules:

• upstream_dict.py
• generation_mix.py
• generation.py

It's worthwhile to go through the processing to see what is used, and what is defunct (if anything) and see about correcting the map between regional generation processes and their respective construction process.

[dt-woods]

Walking through the code, we start in main.py in the run_generation block under include_upstream_processes

>>> upstream_df = get_upstream_process_df(config.model_specs.eia_gen_year)
>>> upstream_df.info()  # note using pandas 2.2.3, so string columns are objects
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 15555423 entries, 0 to 15555422
Data columns (total 24 columns):
 #   Column                      Dtype  
---  ------                      -----  
 0   plant_id                    object 
 1   FuelCategory                object 
 2   stage_code                  object 
 3   FlowName                    object 
 4   Compartment                 object 
 5   Compartment_path            object 
 6   FlowUUID                    object 
 7   Unit                        object 
 8   ElementaryFlowPrimeContext  object 
 9   Source                      object 
 10  Year                        object 
 11  FlowAmount                  object 
 12  quantity                    float64
 13  Electricity                 float64
 14  DataCollection              float64
 15  TemporalCorrelation         float64
 16  GeographicalCorrelation     float64
 17  TechnologicalCorrelation    float64
 18  DataReliability             float64
 19  input                       object 
 20  stage                       object 
 21  FlowType                    object 
 22  Region                      object 
 23  State                       object 
dtypes: float64(7), object(17)
memory usage: 2.8+ GB
>>> upstream_df['FuelCategory'].unique()
array(['CONSTRUCTION', 'NUCLEAR', 'OIL', 'SOLARPV_CONSTRUCTION',
       'SOLARTHERM_CONSTRUCTION', 'WIND_CONSTRUCTION', 'COAL', 'GAS'],
      dtype=object)

From above, we see an inventory data frame where construction processes are identified by their fuel category name.

The following essentially calls olcaschema_genupstream in upstream_dict.py.

>>> upstream_dict = write_upstream_process_database_to_dict(upstream_df)

Examining the keys, we see examples like 'wind_const - Gridforce South'. We can get all these construction process names.

>>> import re
>>> p = re.compile("^(.*)_const - (.*)$")
>>> c_list = [x for x in upstream_dict.keys() if p.match(x)]
>>> c_list[0:5]
['coal_const - Arizona Public Service Company',
 'coal_const - Associated Electric Cooperative, Inc.',
 'coal_const - Bonneville Power Administration',
 'coal_const - California Independent System Operator',
 'coal_const - Dominion Energy South Carolina']

Here's where we start to see some of the problems emerge, for example:

• 'solar_pv_const - Midcontinent Independent Transmission System Operator, Inc..' (two dots at the end of Inc)
• 'ngcc_const - No BA' (not a real region)
• 'ngcc_const - Duke Energy Florida Inc' (no dot after Inc)

Let's dive deeper into olcaschema_genupstream_processes.

Found where regionalized processes were being named!

Created a new method in eia860_facilities.py to align balancing authority names to a data frame that includes plant IDs (also returns a dictionary for further mapping). Probably could just create the dictionary rather than also edit the data frame to increase its utility.

def add_balancing_authorities_to_plants(plant_df, plant_col, year):
    """Helper function to add balancing authority information to a data frame
    with facility IDs.

    Relies on :func:`read_ba_codes` in utils.py to standardized balancing
    authority naming.

    Parameters
    ----------
    plant_df : pandas.DataFrame
        A data frame where ``plant_col`` has facility IDs
    plant_col : str
        The column name associated with plant IDs.
    year : int
        The year to associate with EIA Form 860.

    Returns
    -------
    tuple
        A tuple of length two:

        -   pandas.DataFrame, the original data frame returned with additional
            columns (e.g., 'State', 'NERC Region', 'Balancing Authority Name',
            and 'Balancing Authority Code').
        -   dict, a dictionary with keys associated with plant IDs (int) and
            values associated with balancing authority names (str)

    Notes
    -----
    This method may be better utilized if it extended to multiple years'
    worth of EIA 860 to try to capture as many facilities as possible.
    """

    # Read the EIA Form 860 for the given year
    eia_df = eia860_balancing_authority(year)

    # EIA has 'No BA' for AK facilities; this can cause problems, so remove it!
    noba_filter = eia_df['Balancing Authority Name'] == 'No BA'
    eia_df.loc[noba_filter, 'Balancing Authority Name'] = float('nan')

    # We want BA info, so lose any rows without it.
    eia_df = eia_df.dropna(
        subset=['Balancing Authority Name', 'Balancing Authority Code'],
        how='all'
    ).copy()

    # Standardized BA names
    ba_codes = read_ba_codes()
    eia_df["Balancing Authority Name"] = eia_df["Balancing Authority Code"].map(
        ba_codes['BA_Name'])

    # Make sure plant ID columns are integers for matching;
    # NOTE: you can typically get away with 32-bit integer for plant IDs
    eia_df["Plant Id"] = eia_df["Plant Id"].astype("int32")
    plant_df[plant_col] = plant_df[plant_col].astype("int32")

    # Get sets of plant IDs
    eia_plants = set(eia_df['Plant Id'].tolist())
    user_plants = set(plant_df[plant_col].tolist())

    # Analysis of user plants not found in EIA
    user_only = user_plants - eia_plants
    logging.warning(
        "Found %d plants without balancing authority info!" % len(user_only)
    )

    # Merge the EIA data to the plant data frame.
    plant_df = plant_df.merge(
        eia_df,
        how="left",
        left_on=plant_col,
        right_on="Plant Id"
    ).copy()

    # Create a helper dictionary to map plant IDs to their BA names.
    plant_ba_dict = {
        x[0]: x[1]
        for x in zip(
            eia_df["Plant Id"],
            eia_df["Balancing Authority Name"],
        )
    }

    return (plant_df, plant_ba_dict)

Testing the new method shows that the balancing authority names are now aligned (standardized) to the names used by the processes.

[dt-woods]

Based on test running ELCI_2022, construction processes are now being identified to their regions (note the comma before LLC):