`sustaingym.envs.evcharging`#

Submodules#

Package Contents#

Classes#

`DiscreteActionWrapper`	Discrete action wrapper.
`EVChargingEnv`	EVCharging class.
`MultiAgentEVChargingEnv`	Multi-agent EV charging environment.
`RealTraceGenerator`	Class for `EventQueue` generator using real traces from ACNData.
`GMMsTraceGenerator`	Class for `EventQueue` generator by sampling from trained GMMs.

Functions#

create_gmm(→ None)

Creates a custom GMM and saves in the gmms folder.

Attributes#

DEFAULT_PERIOD_TO_RANGE

class sustaingym.envs.evcharging.DiscreteActionWrapper(env: sustaingym.envs.evcharging.env.EVChargingEnv)[source]#

Bases: gymnasium.ActionWrapper

Discrete action wrapper.

This wrapper maps discrete actions to normalized continuous actions on the EVChargingEnv. Using discrete actions guarantees that non-zero actions will not get zeroed out as in the continuous case (see EVChargingEnv._to_schedule()).

Parameters:: env (sustaingym.envs.evcharging.env.EVChargingEnv) – EV charging environment

action_space#: MultiDiscrete action space

action(act: numpy.ndarray) → numpy.ndarray[source]#

Converts {0, 1, 2, 3, 4} to {0.0, 0.25, 0.5, 0.75, 1.0}.

Parameters:: act (numpy.ndarray) –
Return type:: numpy.ndarray

class sustaingym.envs.evcharging.EVChargingEnv(data_generator: sustaingym.envs.evcharging.event_generation.AbstractTraceGenerator, moer_forecast_steps: int = 36, project_action_in_env: bool = True, verbose: int = 0)[source]#

Bases: gymnasium.Env

EVCharging class.

This classes simulates the charging schedule of electric vehicles (or EVs) connected to an EV charging network. It is based on ACN-Data and ACN-Sim developed at Caltech. Each episode is a 24-hour day of charging, and the simulation can be done using real data from ACN-Data or a Gaussian mixture model (GMM) fitted on the data (see train_gmm_model.py). The gym supports the Caltech and JPL sites.

This environment’s API is known to be compatible with Gymnasium v0.28, v0.29.

In what follows:

n = number of stations in the EV charging network
k = number of steps for the MOER CO2 forecast

Actions:

Type: Box(n)
Action                              Shape   Min     Max
normalized pilot signal             n       0       1

Observations:

Type: Dict(Box(1), Box(n), Box(n), Box(1), Box(k))
                                    Shape   Min     Max
Timestep (fraction of day)          1       0       1
Estimated departures (timesteps)    n       -288    288
Demands (kWh)                       n       0       Max Allowed Energy Request
Previous MOER value                 1       0       1
Forecasted MOER (kg CO2 / kWh)      k       0       1

Parameters:

data_generator (sustaingym.envs.evcharging.event_generation.AbstractTraceGenerator) – generator for sampling EV charging events and MOER forecasts
moer_forecast_steps (int) – number of steps of MOER forecast to include, minimum of 1 and maximum of 36. Each step is 5 mins, for a maximum of 3 hrs.
project_action_in_env (bool) – whether gym should project action to obey network constraints and not overcharge vehicles
verbose (int) –
level of verbosity for print out
- 0: nothing
- 1: print description of current simulation day
- 2: print warnings from network constraint violations and convex optimization solver

# attributes required by gym.Env

action_space#: spaces.Box, structure of actions expected by env

observation_space#: spaces.Dict, structure of observations

reward_range#: tuple[float, float], min and max rewards

spec#: EnvSpec, info about env if initialized from gymnasium.make()

metadata#: dict[str, Any], unused

np_random#: np.random.Generator, random number generator for the env

# attributes specific to EVChargingEnv

data_generator#: AbstractTraceGenerator, generator for sampling EV charging events and MOER forecasting

max_timestep#: int, maximum timestep in a day’s simulation

moer_forecast_steps#: int, number of steps of MOER forecast to include

project_action_in_env#: bool, whether gym should project action to obey network constraints and not overcharge vehicles

verbose#

int, level of verbosity for print out

0: nothing
1: print description of current simulation day
2: print warnings from network constraint violations and convex optimization solver

cn#: acns.ChargingNetwork, EV charging network

num_stations#: int, number of stations in EV charging network

timestep#: int, current timestep in episode, from 0 to 288

TIMESTEP_DURATION = 5#

ACTION_SCALE_FACTOR = 32#

VOLTAGE = 208#

MARGINAL_REVENUE_PER_KWH = 0.15#

OPERATING_MARGIN = 0.2#

MARGINAL_PROFIT_PER_KWH#

CO2_COST_PER_METRIC_TON = 30.85#

A_MINS_TO_KWH#

VIOLATION_WEIGHT = 0.001#

A_PERS_TO_KWH#

PROFIT_FACTOR#

VIOLATION_FACTOR#

CARBON_COST_FACTOR#

step(action: numpy.ndarray) → tuple[dict[str, numpy.ndarray], float, bool, bool, dict[str, Any]][source]#

Steps the environment.

Calls the step function of the internal simulator.

Parameters:

action (numpy.ndarray) – shape [num_stations], normalized charging rate for each charging station between 0 and 1.

Returns:

observation – state
- ‘est_departures’: shape [num_stations], the estimated number of periods until departure. If there is no EVSE at the index, the entry is set to zero.
- ‘demands’: shape [num_stations], amount of charge demanded by each EVSE in kWh.
- ‘prev_moer’: shape [1], emissions rate for the current timestep in kg CO2 per kWh. Between 0 and 1.
- ‘forecasted_moer’: shape [moer_forecast_steps], forecasted emissions rate for next timestep(s) in kg CO2 per kWh. Between 0 and 1.
- ‘timestep’: shape [1], fraction of day between 0 and 1.
reward – scheduler’s performance metric per timestep
terminated – whether episode is terminated
truncated – always False, since there is no intermediate stopping condition
info – auxiliary useful information
- ‘num_evs’: int, number of charging sessions in episode.
- ‘avg_plugin_time’: float, average plugin time in periods (5 mins) across sessions in episode.
- ‘max_profit’: float, maximum profit if all EVs were charged maximally while they are connected to the network. This does not take into account network constraints or carbon emissions, and it is a good proxy for info[‘reward_breakdown’][‘profit’].
- ‘reward_breakdown’: dict[str, float], breakdown of evaluation metrics cumulative over the episode.
  - ‘profit’ ($) : profit over charge delivered to all EVs.
  - ‘carbon_cost’($): cost of marginal emissions.
  - ‘excess_charge’ ($): cost of network violations.
- ‘evs’: list[acnm.ev.EV], list of EVs in the event queue
- ‘active_evs’: list[acnm.ev.EV], list of active EVs at current timestep
- ‘moer’: array, shape [289, 37] emissions rate for entire episode.
- ‘pilot_signals’: DataFrame, pilot signals received by simulator

Return type:

tuple[dict[str, numpy.ndarray], float, bool, bool, dict[str, Any]]

reset(*, seed: int | None = None, options: dict[str, Any] | None = None) → tuple[dict[str, numpy.ndarray], dict[str, Any]][source]#

Resets the environment.

Prepares for the next episode by re-creating the charging network, generating new events, creating the simulation and interface, and resetting information-tracking variables.

Parameters:

seed (int | None) – seed for resetting the environment. An episode is entirely reproducible no matter the generator used.
options (dict[str, Any] | None) –
resetting options
- ’verbose’: set verbosity level [0-2]

Returns:

observation – state dict, see step()
info – info dict, see step()

Return type:

tuple[dict[str, numpy.ndarray], dict[str, Any]]

close() → None[source]#

Close the environment. Delete internal variables.

Return type:: None

class sustaingym.envs.evcharging.MultiAgentEVChargingEnv(data_generator: sustaingym.envs.evcharging.event_generation.AbstractTraceGenerator, periods_delay: int = 0, moer_forecast_steps: int = 36, project_action_in_env: bool = True, discrete: bool = False, verbose: int = 0)[source]#

Bases: pettingzoo.ParallelEnv

Multi-agent EV charging environment.

Each charging station is modeled as an independent agent with a single action of the pilot signal to supply.

This environment’s API is known to be compatible with PettingZoo v1.24.1

Observations for each agent are flattened.

Parameters:

data_generator (sustaingym.envs.evcharging.event_generation.AbstractTraceGenerator) –
periods_delay (int) –
moer_forecast_steps (int) –
project_action_in_env (bool) –
discrete (bool) –
verbose (int) –

# attributes required by pettingzoo.ParallelEnv

agents#: list[str], agent IDs (which are the charging station IDs)

possible_agents#: list[str], same as agents

observation_spaces#: dict[str, spaces.Box], observation space for each agent

action_spaces#: dict[str, spaces.Box], action space for each agent

# attributes specific to MultiAgentEVChargingEnv

single_env#: EVChargingEnv, single-agent EVChargingEnv

periods_delay#: int, time periods of delay for inter-agent communication

metadata#

step(actions: dict[str, numpy.ndarray]) → tuple[dict[str, numpy.ndarray], dict[str, float], dict[str, bool], dict[str, bool], dict[str, dict[str, Any]]][source]#

Returns:

obss – dict mapping agent_id to observation
rewards – dict mapping agent_id to reward
terminateds – dict mapping agent_id to terminated
truncateds – dict mapping agent_id to truncated
infos – dict mapping agent_id to info

Parameters:

actions (dict[str, numpy.ndarray]) –

Return type:

tuple[dict[str, numpy.ndarray], dict[str, float], dict[str, bool], dict[str, bool], dict[str, dict[str, Any]]]

reset(seed: int | None = None, options: dict | None = None) → tuple[dict[str, numpy.ndarray], dict[str, numpy.ndarray]][source]#

Resets the environment.

Parameters:

seed (int | None) –
options (dict | None) –

Return type:

tuple[dict[str, numpy.ndarray], dict[str, numpy.ndarray]]

render() → None[source]#

Render environment.

Return type:: None

close() → None[source]#

Close the environment.

Return type:: None

observation_space(agent: str) → gymnasium.spaces.Space[source]#

Parameters:: agent (str) –
Return type:: gymnasium.spaces.Space

action_space(agent: str) → gymnasium.spaces.Box | gymnasium.spaces.Discrete[source]#

Parameters:: agent (str) –
Return type:: gymnasium.spaces.Box | gymnasium.spaces.Discrete

class sustaingym.envs.evcharging.RealTraceGenerator(site: sustaingym.envs.evcharging.utils.SiteStr, date_period: tuple[str, str] | sustaingym.envs.evcharging.utils.DefaultPeriodStr, sequential: bool = True, use_unclaimed: bool = False, requested_energy_cap: float = 100, seed: int | None = None)[source]#

Bases: AbstractTraceGenerator

Class for EventQueue generator using real traces from ACNData.

See AbstractTraceGenerator for more arguments and attributes

Parameters:

sequential (bool) – whether to draw simulated days sequentially from date range or randomly
use_unclaimed (bool) – whether to use unclaimed sessions, which do not have the “requested energy” or “estimated departure” attributes. If True, the generator uses the energy delivered in the session and the disconnect time in place of those attributes, eliminating real-world uncertainty in user requests.
seed (int | None) – if sequential, the seed determines which day to start on
site (sustaingym.envs.evcharging.utils.SiteStr) –
date_period (tuple[str, str] | sustaingym.envs.evcharging.utils.DefaultPeriodStr) –
requested_energy_cap (float) –

sequential#: whether to draw simulated days sequentially from date range or randomly

use_unclaimed#: whether to use unclaimed sessions, which do not have the “requested energy” or “estimated departure” attributes. If True, the generator uses the energy delivered in the session and the disconnect time in place of those attributes, eliminating real-world uncertainty in user requests.

set_seed(seed: int | None) → None[source]#

If days are sequential, sets the day. Otherwise, seeds the random number generator.

Parameters:: seed (int | None) –
Return type:: None

class sustaingym.envs.evcharging.GMMsTraceGenerator(site: sustaingym.envs.evcharging.utils.SiteStr, date_period: tuple[str, str] | sustaingym.envs.evcharging.utils.DefaultPeriodStr, n_components: int = 30, requested_energy_cap: float = 100, seed: int | None = None)[source]#

Bases: AbstractTraceGenerator

Class for EventQueue generator by sampling from trained GMMs.

See AbstractTraceGenerator for more arguments and attributes

Parameters:

site (sustaingym.envs.evcharging.utils.SiteStr) – garage to get events from, either ‘caltech’ or ‘jpl’
date_period (tuple[str, str] | sustaingym.envs.evcharging.utils.DefaultPeriodStr) – either a pre-defined date period or a custom date period. If custom, the input must be a 2-tuple of strings with both strings in the format YYYY-MM-DD. Otherwise, should be a default period string.
n_components (int) – number of components in GMM
requested_energy_cap (float) – max amount of requested energy allowed (kWh)
seed (int | None) – seed for random sampling

n_components#: int, number of components in use for GMM

gmm#: sklearn.mixture.GaussianMixture, models sessions distribution

cnt#: np.ndarray, shape [num_days], empirical distribution for number of sessions on each day

station_usage#: np.ndarray, shape [num_stations], total number of sessions during interval for each station

Notes about saved GMMs

default gmm directory: in package sustaingym/data/evcharging/gmms
    gmms
    |----caltech
    |   |---2019-05-01 2019-08-31 30.pkl
    |   |---2019-09-01 2019-12-31 30.pkl
    |   |---2020-02-01 2020-05-31 30.pkl
    |   |---2021-05-01 2021-08-31 30.pkl
    |----jpl
    |   |---2019-05-01 2019-08-31 30.pkl
    |   |---2019-09-01 2019-12-31 30.pkl
    |   |---2020-02-01 2020-05-31 30.pkl
    |   |---2021-05-01 2021-08-31 30.pkl
    Each '*.pkl' file containing a trained GMM, station usage count,
        and daily session count.
custom gmm directory: GMMs can also be trained on custom date ranges
    and number components. These are saved in the 'gmms' folder
    relative to the current working directory. See train_gmm_model.py
    for how to train GMMs from the command line.

set_seed(seed: int | None) → None[source]#

Sets random seed to make GMM sampling reproducible.

Parameters:: seed (int | None) –
Return type:: None

sustaingym.envs.evcharging.create_gmm(site: sustaingym.envs.evcharging.utils.SiteStr, n_components: int, date_range: tuple[datetime.datetime, datetime.datetime]) → None[source]#

Creates a custom GMM and saves in the gmms folder.

Parameters:

site (sustaingym.envs.evcharging.utils.SiteStr) – either ‘caltech’ or ‘jpl’
n_components (int) – number of components of Gaussian mixture model
date_range (tuple[datetime.datetime, datetime.datetime]) – a range of dates that falls inside 2018-11-01 and 2021-08-31.

Return type:

None

sustaingym.envs.evcharging.DEFAULT_PERIOD_TO_RANGE[source]#

sustaingym.envs.evcharging#

Submodules#

Package Contents#

Classes#

Functions#

Attributes#

`sustaingym.envs.evcharging`#