Introduction

Dexterous Developer is a modular hot-reload system for Rust. At this point, it provides an adapter for Bevy - but more adapters will be added in the near future.

Features

• A CLI for building & running reloadable rust projects, including over the network (cross-device)
• The ability to serialize/deserialize elements, allowing the evolution of schemas over time
• Only includes any hot reload capacity in your build when you explicitly enable it - such as by using the CLI launcher
• The capacity to create adapters for additional frameworks, allowing you to use Dexterous Developer tooling with other tools.
• Includes a first-party Bevy adapter
• Works on Windows, Linux, and MacOS
• On Linux, can be used to develop within a dev container while running on the main OS, enabling use of dev containers for games & other GUI apps.

Bevy Specific

• Define the reloadable areas of your game explicitly - which can include systems, components, states, events and resources (w/ some limitations)
• Reset resources to a default or pre-determined value upon reload
• Serialize/deserialize your reloadable resources & components, allowing you to evolve their schemas so long as they are compatible with the de-serializer (using rmp_serde)
• Mark entities to get removed on hot reload
• Run systems after hot-reload
• Create functions to set up & tear down upon either entering/exiting a state or on hot reload

Inspiration

This project was inspired by DGriffin91's Ridiculous bevy hot reloading.

How Dexterous Developer Works

Dexterous Developer is designed to be a modular hot reload system, and it does so by separating into a set of components and standerdizing the communication between them.

If you don't need additional customization, you only need to care about the adapter for your framework or the dexterous_developer_instance crate if no adapter exists, along with using the provided CLIs.

Instance Side Components

These components run within the reloaded codebase, and provide the the main point of integration with your own code.

• Instances - these provide the lower-level elements that the adapters build on, such as providing hooks for knowing an update is available, replacing the current running instance, and ways to call elements through the reload boundary.
• Adapters - these are build specifically to embed within another framework or known pattern for managing a persistent application. Hot Reload only makes sense in long-running processes, ideally ones that repeat regularly such as UI, Graphics, Simulations, and Web Servers. As a rule, they often have some form of loop or trigger-based execution pattern, a way to persist state across executions, and some known methods of interacting with the outside world. The Adapter provides a way to integrate directly with a specitic framework, while providing some higher-level features on top. At this time, bevy_dexterous_developer is the only first-party adapter we provide.

Client Side Components

These components are used to execute the instance, download updates to the libraries or assets, and basically any element that needs to be managed externally to your application and not re-loaded along with it.

• Runners - these provide the necessary logic to handle executing the application using a particular reload approach. At this time, dynamic libraries are the only supported approach - provided by dexterous_deverloper_dylib_runner.
• The Runner CLI - this provides a pre-made way to execute reloadable applications via the command line, so you don't need to set up a runner in your own codebase.

Server Side Components

These components are used to watch the codebase for changes, manage builds, and provide the runners with access to the associated code and assets.

• Builders - these handle triggering the cargo builds themselves, processing the output, and collecting library dependencies.
  • Rustc Wrapper - the default builder utilizes a wrapper around Rustc, to enable some capabilities such as the ability to work with binary crates.
• Watchers - these handle watching the file system for changes, allowing for notifications in changes to assets (non-compiled files) and sending out the triggers for re-builds when the code itself changes.
• Manager - the manager handles setting up the chosen watcher & builders for each available target, and sets up a server for the runner to connect to.
• The Server CLI - this provides a pre-made way to run a build server using the default builders & watcher.

Shared Types

The dexterous_developer_types crate provides types that are shared across components, both on the client and the server.

Bevy

Bevy is an open-source game engine built in Rust, and is the origianl usecase for Dexterous Developer. As a result, it's adapter provides a wide set of supported features.

Supported Features

• System Replacement and Registration
• Reloading and Resetting Resources
• Reloading Components and Re-Running Setup Functions
• Replacing Events
• Reloading States

Warning

Note that you cannot use Resources, Components, Events or States registered in a reloadable scope outside of the elements systems tied to that scope. Otherwise, you run the risk of running into undefined behaviour.

For example:

#![allow(unused)]
fn main() {
reloadable_main!((initial_plugins) {
    App::new()
        .add_plugins(initial_plugins.initialize::<DefaultPlugins>())
        .setup_reloadable_elements::<reloadable>()
         // This will fail after reload since it queries MyComponent, which
         // is registered as a serializable component.
        .add_systems(Update, count_components)
        
        .run();
});

#[derive(Component, Serialize, Deserialize)]
struct MyComponent;

impl SerializableType for MyComponent {
    fn get_type_name() -> &'static str {
        "MyComponent"
    }
}

reloadable_scope!(reloadable(app) {
    app
        .register_serializable_component::<Myomponent>()
        // Instead, place any systems that rely on elements that are reloadable
        // within the reloadable scope.
        .add_systems(Update, count_components);
})

fn count_components(query: Query<&MyComponent>) {
    /../
}

}

This is because reloadable elements will be replaced with potentailly different types, but the systems registered in reloadable_main will keep using the old version.

Bevy Quick Start

Installation

Install the CLI by running: cargo install dexterous_developer_cli. This installs 2 command line utilities:

• dexterous_developer_cli - used to build the project, potentially running it at the same time
• dexterous_developer_runner - used to run the project on another device

Note for pre-relesse versions

Make sure that the version of dexterous_developer_cli matches the version you are installing. While the goal is to eventually have more separation between the two, for now they should be kept in sync.

General Setup

Dynamic Libraries

Previous versions of Dexterous Developer required you to make your crate a dynamic library. That is no longer necessary!

You can still use an explicit dynamic library as your main crate just like before, but you can also place the reloadable_main! macro in main.rs, and not provide it with a function name - it will default to "main":

#![allow(unused)]
fn main() {
reloadable_main!((initial_plugins) {
  App::new()
  ...
})
}

You'll also need to add the appropriate dexterous developer adapter to your library's dependencies, and set up the "hot" feature. For example, if you are using bevy:

[features]
hot = ["bevy_dexterous_developer/hot"]

[dependencies]
bevy = "0.14"
bevy_dexterous_developer = { version = "0.4.0-alpha.3" }
serde = "1" # If you want the serialization capacities

Finally, you'll need to set up a Dexterous.toml file`

features = [
    "hot"
]
code_watch_folders = ["./src"]
asset_folders = ["./assets"]

Bevy Code

In main.rs, wrap your main function with the reloadable_main macro:

#![allow(unused)]
fn main() {
reloadable_main!((initial_plugins) {
    App::new()
        .add_plugins(initial_plugins.initialize::<DefaultPlugins>()) // You can use either DefaultPlugins or MinimnalPlugins here, and use "set" on this as you would with them
    // Here you can do what you'd normally do with app
    // ... and so on
});
}

And wherever you want to add reloadable elements - such as systems, components, or resources - to your app, do so within a reloadable_scope! macro - and add the reloadable scope to the app using setup_reloadable_elements:

#![allow(unused)]

fn main() {
impl Plugin for MyPlugin {
    fn build(&self, app: &mut App) {
        app
            .setup_reloadable_elements::<reloadable>();
    }
}

reloadable_scope!(reloadable(app) {
    app
        .add_systems(Update, this_system_will_reload);
})
}

Running with Hot Reload

To run a hot-reloaded app locally, cargo install and run dexterous_developer_cli (optionally passing in a specific package or example).

To run the app on a different machine (with the same platform), cargo install dexterous_developer_cli on both machines, and then:

• run the dexterous_developer_cli --serve-only on the development machine
• run the dexterous_developer_runner --server http://*.*.*.*:4321 command, ideally in a dedicated directory, on the target machine

Running or Building Without Hot Reload

Once you have everything set up for development, you will likely want to be able to build production versions of the application as well. This will require creating a separate binary. To do so, you can add a bins/launcher.rs to your project:

fn main() {
    PACKAGE_NAME::bevy_main();
}

and in your Cargo.toml, you'll need to add:

[[bin]]
name = "launcher"
path = "bins/launcher.rs"

You can then run the non-hot-reloadable version of your app using cargo run --bin launcher (or build with cargo build --bin launcher). Remember to avoid including the hot feature, since it's designed to work only inside a reloadable library!.

System Replacement and Registration

The simplest feature of dexterous_developer is the ability to register, remove and replace systems. To do so, all you need to do is use .add_systems within a reloadable_scope!:

#![allow(unused)]
fn main() {
    reloadable_scope!(reloadable(app) {
        app
            .add_systems(Update, my_dope_system);
    });
}

Any system added from within a reloadable scope will be removed before the reloadable scope function runs to set up the replacement systems.

Resources

Resources can be reloaded in a variety of ways.

Resetting Resources

If you have a resource that you want to re-set when a reload occurs, you can do so using either app.reset_resource::<R: Resource + Default>() or app.reset_resource_to_value::<R: Resource>(value: R) within a reloadable scope. This will cause the resource to be removed and re-initialized when new coad is loaded.

Serializable Resources

If you have a resource that you want to serialize and de-serialize, allowing you to maintain it's state while evolving it's schema.

You initialize the resource by using either app.init_serializable_resource::<R: ReplacableType + Resource + Default>() or app.insert_serializable_resource::<R: ReplacableType + Resource>(initializer: impl 'static + Send + Sync + Fn() -> R)

You can also mark a resource type as serializable without actively adding a copy of it to the application using app.register_serializable_resource::<R: ReplacableType + Resource>(). This will only serialize/deserialize the resource if it existins at the time of the reload.

• using serde and implementing SerializableResource. This approach relies on rmp_serde to serialize and deserialize the resource.

  #![allow(unused)]
  fn main() {
    #[derive(Resource, Serialize, Deserialize)]
    struct MyResource(String);
  
    impl SerializableType for MyResource {
        fn get_type_name() -> &'static str {
            "MyResource
        }
    }
  }

• implementing ReplacableType yourself:

  #![allow(unused)]
  fn main() {
  #[derive(Resource)]
  struct MyResource(String);
  
  impl ReplacableType for MyResource {
    fn get_type_name() -> &'static str {
        "MyResource"
    }
  
    fn to_vec(&self) -> bevy_dexterous_developer::Result<Vec<u8>> {
        Ok(self.0.as_bytes().to_vec())
    }
  
    fn from_slice(val: &[u8]) -> bevy_dexterous_developer::Result<Self> {
        Ok(Self(std::str::from_utf8(val))?))
    }
  }
  }

Components

Components can be handled in a few ways as well.

Serializable Components

If you have a component that you want to serialize and de-serialize, allowing you to maintain it's state while evolving it's schema.

You set up the component as serializable by calling app.register_serializable_component<C: Component + ReplacableType>() within a reloadable scope.

• using serde and implementing SerializableType. This approach relies on rmp_serde to serialize and deserialize the resource.

  #![allow(unused)]
  fn main() {
    #[derive(Component, Serialize, Deserialize)]
    struct MyComponent(String);
  
    impl SerializableType for MyComponent {
        fn get_type_name() -> &'static str {
            "MyComponent
        }
    }
  }

• implementing ReplacableType yourself:

  #![allow(unused)]
  fn main() {
  #[derive(Component)]
  struct MyComponent(String);
  
  impl ReplacableType for MyComponent {
    fn get_type_name() -> &'static str {
        "MyComponent"
    }
  
    fn to_vec(&self) -> bevy_dexterous_developer::Result<Vec<u8>> {
        Ok(self.0.as_bytes().to_vec())
    }
  
    fn from_slice(val: &[u8]) -> bevy_dexterous_developer::Result<Self> {
        Ok(Self(std::str::from_utf8(val))?))
    }
  }
  }

Clear on Reload

Alternatively, you may want to fully remove any entities that have a given component upon reload. To do so, you just need to call app.clear_marked_on_reload::<C: Component>() from within a reloadable scope. Whenever a reload occurs, all entities with the component will be removed. Note - this will also despawn any descendents.

Reset Setup

Finally, you may wish to both clear all entities with a component and run a setup function after reload - for example, to re-build a UI. You can do so in one of 2 ways - calling app.reset_setup<C: Component, M>(systems), which will clear all entities with the component (and their descendents) and then run the systems on every reload, or calling app.reset_setup_in_state<C: Component, S: States, M>(state: S, systems) which will despawn on every reload, but only run the setup if you are in a given state. This will despawn marked systems on exit, similar to enable_state_scoped_entities.

Replacing Events

Since events are ephemeral, there is no value in maintaining them as serializable elements like we do resources, components or states.

However, there can still be value in being able to register new events or replace old ones. To do so, simply use app.add_event<E: Event>() from within a reloadable scope. The usage here is identical to standard bevy events. Note that upon a reload, the event queue will be fully replaced - so you will miss any events issued during the previous frame.

States

Setting Up States

The states you define within a reloadable scope need to be either SerializableType or ReplacableType - but otherwise their behaviour should match native bevy states. Note that no transition gets triggered upon the initial change - so if you changed the logic for a sub states existance or a computed state's compute function you may need to trigger the transitions manually..

For Freely Mutable States, call app.init_state<S: FreelyMutableState + ReplacableType + Default>() or app.insert_state<S: FreelyMutableState + ReplacableType>(initial: S) within a reloadable scope.

For Sub States, call app.add_sub_state<S: SubStates + ReplacableTypes>(), and for Computed States call app.add_computed_state<S: ComputedState + ReplacableTypes>().

![Note] ReplacableTypes is required for computed states to avoid having to re-trigger transitions while retaining the current value. Otherwise, we would need to Exit all reloadable states before a reload and re-enter the new states after - which could re-set elements of state you want to keep. Instead, handle any such re-sets with app.reset_setup_in_state with the help of marker components.

You can either implement SerializableType:

#![allow(unused)]

fn main() {
#[derive(States, Debug, Default, Hash, PartialEq, Eq, Clone, Serialize, Deserialize)]
enum MyState {
    #[default]
    InitialState,
    AnotherState
}

impl SerializableType for MyState {
    fn get_type_name() -> &'static str {
        "MySerializableResource"
    }
}
}

or ReplacableType directly:

#![allow(unused)]

fn main() {
#[derive(States, Debug, Default, Hash, PartialEq, Eq, Clone)]
enum MyState {
    #[default]
    InitialState,
    AnotherState
}

impl ReplacableType for MyState {
    fn get_type_name() -> &'static str {
        "MySerializableResource"
    }

    fn to_vec(&self) -> bevy_dexterous_developer::Result<Vec<u8>> {
        let value = match self {
            MyState::InitialState => [0],
            MyState::AnotherState => [1],
        };
        Ok(value.to_vec())
    }

    fn from_slice(val: &[u8]) -> bevy_dexterous_developer::Result<Self> {
        let value = if let Some(val) = val.get(0) {
            if *val == 1 {
                MyState::AnotherState
            } else {
                MyState::InitialState
            }
        } else {
            MyState::InitialState
        };
        Ok(value)
    }
}
}

State Scoped Entities

For most contexts, you will want to use reset_setup_in_state<C: Component, S: States, M>(state: S, systems) instead - this combines running systems in OnEnter or upon reload, and despawning entities marked with C (and their descendents) OnExit or OnReload.

However, if you want a Scoped entity that doesn't despwan, but instead remains so long as you are in the given state across reloads, you can also use enable_state_scoped_entities<S: States + ReplacableType>() just as you would outside the reloadable scope, and then add StateScoped(value: S) to any entities you care about here.

Github Repository

Crates.io