The Transfer Rework

Introduction

The release of NeoForge 21.9.1-beta comes with a fundamental redesign of the item, fluid and energy transfer systems, with the goal of fixing many issues that were found in the previous iteration after years of usage.

This change affects the IItemHandler, IFluidHandler, IFluidHandlerItem and IEnergyStorage interfaces, the corresponding capabilities, and all the supporting classes. The new system will be introduced in the first part of this post.

Since this is a large breaking change, steps have been taken to soften the transition, as will be outlined below in the second part of this post.

The new transfer APIs

Here are the replacement for the old APIs:

  • IItemHandler: replaced by ResourceHandler<ItemResource>.
  • IFluidHandler: replaced by ResourceHandler<FluidResource>.
  • IFluidHandlerItem: replaced by ResourceHandler<FluidResource> + a suitable ItemAccess passed to the capability query.
  • IEnergyStorage: replaced by EnergyHandler.

Resources

Resources are objects that represent what is being transferred. For example, an ItemResource consists of an Item as well as an immutable component map. Resources allows separating what is being transferred and how much is being transferred, as will be shown below.

Resources have the following properties:

  • Immutability: resources are immutable objects.
  • No amount: resources do not contain information about how much is being transferred.
  • Overrides for equals and hashCode: resources are compared with equals, and can be hashed with hashCode. Combined with their immutability, this makes them great as map keys.
  • Implement Resource: all resources implement Resource, which provides the isEmpty() function.

Currently, NeoForge provides two resource implementations: ItemResource and FluidResource, which work in very similar ways. Here are some usage examples of ItemResource:

// Construction of a resource from an item:
ItemResource resource = ItemResource.of(Items.DIAMOND);

// Conversion of a stack to a resource:
ItemStack stack = /* ... */;
ItemResource resource = ItemResource.of(stack);

// Conversion of a resource to a stack:
ItemStack stack = resource.toStack();
ItemStack stack10 = resource.toStack(10); // optional amount

// Comparison of resources:
resource1.equals(resource2)

// Usage in map keys:
Object2IntMap<ItemResource> amounts = new OpenObject2IntHashMap<>();
amounts.put(ItemResource.of(Items.APPLE), 10); // 10 apples

NeoForge also provides a ResourceStack, which is a simple immutable wrapper around a resource and an amount.

Resource handlers

The core interface for resource transfer is ResourceHandler<T extends Resource>, exposing the following functionality:

  • Handler metadata querying:
    • size
    • isValid
    • getCapacityAsLong, or getCapacityAsInt for usage in int contexts
  • Handler contents querying:
    • getResource
    • getAmountAsLong, or getAmountAsInt for usage in int contexts
  • Handler modifications:
    • insert
    • extract

Note that the insert and extract methods come in both a version working on a specific index of the handler, as well as a version that works over the entire handler. The version that works over the entire handler should be preferred if possible as it is more convenient and lets the handler potentially implement the operation much more effectively.

The insert and extract methods have the same signature. Here is for example the slotted insert method:

// Returns: how much was inserted
int insert(int index, T resource, int amount, TransactionContext transaction);

Here is a breakdown:

  • The int index parameter idifies the slot of the handler, and should be between 0 (inclusive) and size() (exclusive).
  • The T resource parameter indentifies which resource is being inserted.
  • The int amount parameter indicates how much of the resource is being inserted.
  • The TransactionContext transaction parameter is the transaction for this operation. (see below)
  • The method returns how much was accepted, i.e. how much was inserted for insert and how much was extracted for extract. If the resource cannot be inserted/extracted, the method should return 0.

Transactions

The new transfer APIs are built around the concept of transactions, which contain multiple individual operations and can be either executed or cancelled at once. You can think of transactions as video game checkpoints:

  • Opening a transaction creates a new checkpoint.
  • Committing a transaction validates the change, discarding the checkpoint.
  • Aborting a transaction restores the state to the previous checkpoint.

Transactions are opened with Transaction.open(@Nullable parent), always using a try-with-resources statement. The transaction is confirmed using the .commit() method - if the transaction was not committed by the end of the try block, it will be aborted automatically.

Here is an example of inserting exactly 10 apples in a resource handler:

boolean insert10Apples(ResourceHandler<ItemResource> handler) {
    // Open a new transaction:
    try (Transaction tx = Transaction.open(null)) {
        // Insert up to 10 apples
        if (handler.insert(ItemResource.of(Items.APPLE), 10, tx) == 10) {
            // If 10 apples were inserted (return value 10), then commit the transaction.
            tx.commit();
            return true;
        }
        return false;
        // The transaction was not committed, so here the insertion will be reverted.
    }
}

Here is another example, converting 16 coal items into 1 diamond:

// Extracts 16 coal from slot 0 and inserts 1 diamond into slot 1. Only if both succeed.
// Returns true if both operations succeeded, false otherwise.
boolean coalToDiamonds(ResourceHandler<ItemResource> handler, boolean simulate) {
    var coal = ItemResource.of(Items.COAL);
    var diamond = ItemResource.of(Items.DIAMOND);

    // Open transaction:
    try (var tx = Transaction.openRoot()) {
        if (handler.extract(0, coal, 16, tx) != 16) {
            // If we can't extract 16 coal, abort immediately.
            // This will revert the extraction of the coal.
            return false;
        }
        if (handler.insert(1, diamond, 1, tx) != 1) {
            // If we can't insert 1 diamond, abort immediately.
            // This will also revert the extraction of the 16 coal items.
            return false;
        }
        // By now we know that the operation is a success, so we will return true.
        // If we are just "simulating" the operation, we don't commit to revert the change anyway.
        if (!simulate) {
            tx.commit();
        }
        return true;
    }
}

This is implemented internally by saving state snapshots, and reverting to them if necessary. The bulk of the work is handled by the SnapshotJournal class, which needs to be subclassed for transaction support. In most cases, using the provided implementations, means that no additional work is needed to support transactions.

Provided implementations

NeoForge provides an extensive set of ResourceHandler implementations that will cover most needs, and serve as a reference for modders who want to write their own ResourceHandler implementations. Here is an overview.

Classes that support any resource type:

  • CombinedResourceHandler: chains multiple handlers together.
  • DelegatingResourceHandler: base implementation that delegates all methods to another handler.
  • EmptyResourceHandler: empty resource handler.
  • InfiniteResourceHandler: contains an infinite amount of a single resource.
  • RangedResourceHandler: limits a resource handler to either an index range or a single index.
  • StacksResourceHandler: base implementation of a resource handler that stores its contents in a list of stacks.
    • ResourceStacksResourceHandler: for the special case of a list of ResourceStacks.
  • VoidingResourceHandler: resource handler that voids any inserted resource.

Classes specific to fluids:

  • FluidStacksResourceHandler: base implementation of a resource handler that stores its contents in a list of FluidStacks.

Classes specific to items:

  • CarriedSlotWrapper: wraps the carried slot of a menu.
  • ItemStackResourceHandler: base implementation of a single-index handler backed by an ItemStack.
  • ItemStacksResourceHandler: base implementation of a resource handler that stores its contents in a list of ItemStacks.
  • LivingEntityEquipmentWrapper: wraps the armor or hands of a living entity.
  • PlayerInventoryWrapper: wraps a player’s inventory. Comes with many useful helpers as well.
  • VanillaContainerWrapper: wraps a vanilla Container.
  • WorldlyContainerWrapper: wraps a vanilla WorldlyContainer.

Provided helpers

Additionally, make sure to check ResourceHandlerUtil which contains many powerful helpers. My favorite one is ResourceHandlerUtil.move(from, to, ...) which streamlines moving resources between two handlers.

For fluids and items, check FluidUtil (the new one) and ItemUtil as well.

Item access

Item capabilities, as used by buckets, bundles, and battery items, now use an ItemAccess context. An ItemAccess provides access to an item storage location, like a slot in an inventory or a player’s hand such that the current item resource and amount can be read, and the stored item can be changed. This allows the containing inventory to be updated directly, while a user of the capability only needs to interact with a normal ResourceHandler that.

Here is an example of directly adding a bucket of water into the item that the player is currently holding:

Player player = /* ... */;
// Create an item access for the player's hand
ItemAccess itemAccess = ItemAccess.forPlayerInteraction(player, InteractionHand.MAIN_HAND)
        // And limit the access to a single item at the time
        .oneByOne();
// Get the item capability for fluid handlers
ResourceHandler<FluidResource> handler = itemAccess.getCapability(Capabilities.Fluid.ITEM);
if (handler == null) {
    // No handler, return false
    return false;
}
// Once we have obtained the handler, we use it like any other handler.
// Modifications to the player's inventory such as filling and "stowing" buckets will be performed automatically.
// For example, we can insert some water:
try (var tx = Transaction.open(null)) {
    if (handler.insert(FluidResource.of(Fluids.WATER), FluidType.BUCKET_VOLUME, tx) == FluidType.BUCKET_VOLUME) {
      // If the insertion was successful, commit and return true
      tx.commit();
      return true;
    }
}
return false;

Custom instances of ItemAccess can be created, however NeoForge provides implementations for many typical usages, such as:

  • ItemAccess.forPlayerInteraction: for interactions with a player’s hand.
  • ItemAccess.forPlayerCursor: for menu interactions with the cursor.
  • ItemAccess.forPlayerSlot: for interactions with a specific slot of a player’s inventory.
  • ItemAccess.forHandlerIndex and ItemAccess.forHandlerIndexStrict: for interaction with a slot of a resource handler.
  • ItemAccess.forStack: to mutate an item stack directly.

Additionally, NeoForge provides base implementations for resource handlers that are backed by an ItemAccess:

  • ItemAccessResourceHandler: flexible base implementation, works for any resource type.
  • ItemAccessFluidHandler: fluid base implementation, stores data in a SimpleFluidContent component.
  • ItemAccessItemHandler: item base implementation, stores data in an ItemContainerContents component.

Energy

Since energy does not have a concept of resources, it has its own simpler EnergyHandler interface with the following methods:

  • getCapacityAsLong / getCapacityAsInt
  • getAmountAsLong / getAmountAsInt
  • insert
  • extract The main differences compared to the old IEnergyStorage are transaction support, method renames, and optional long getters.

This new energy interface comes with many corresponding helpers. I invite you to go through the classes in the net.neoforged.neoforge.transfer.energy package:

  • DelegatingEnergyHandler: base implementation that delegates all methods to another handler.
  • EmptyEnergyHandler: empty energy handler.
  • EnergyHandler: the new energy handler interface.
  • EnergyHandlerUtil: utility class with useful methods such as getRedstoneSignalFromEnergyHandler and move.
  • InfiniteEnergyHandler: contains an infinite amount of energy.
  • ItemAccessEnergyHandler: base implementation of an energy handler for item stacks, stores data in an Integer data component.
  • LimitingEnergyHandler: wraps an energy handler with additional per-insert and per-extract limits.
  • SimpleEnergyHandler: base implementation of a simple energy handler.
  • VoidingEnergyHandler: energy handler that voids any inserted energy.

How to migrate

I am aware that many modders are dealing with changes in the transfer handling at the same time as many other changes, and would rather not rewrite their mod entirely.

It is possible to first port to NeoForge 21.9.0-beta which does not include the transfer rework. Alternatively, porting straight to any later version of NeoForge will combine vanilla changes with the transfer rework. Due to the transactional nature of the new transfer API, it is not possible to provide a wrapper that turns any IItemHandler into an ResourceHandler<ItemResource>, and similarly for the other resource types. The capabilities for the old handlers have also been replaced by capabilities for the new handlers. This means that mods who register capability providers will need to migrate, however read on.

Wrapping in the reverse direction is possible, and provided by NeoForge:

  • IItemHandler.of(handler) will wrap a ResourceHandler<ItemResource> as a an IItemHandler.
  • IFluidHandler.of(handler) will wrap a ResourceHandler<FluidResource> as a IFluidHandler.
  • IEnergyStorage.of(handler) will wrap an EnergyHandler as an IEnergyStorage.

While the capabilities for the old handlers have been replaced, the old handlers themselves have been deprecated for removal but left mostly unchanged. Combined with the above adapters, this means that mods who use the capabilities will need minimal changes to make their mod work again.

This allows the following migration strategy.

Step 1: Making your mod compile again

As long as your mod doesn’t compile, it is impossible to test any change, making upgrading much more difficult. So the first step is make your mod compile again.

1.1: Comment any capability provider registration

This means that your mod’s containers, tanks, etc… won’t be recognized as item/fluid/energy handlers for the time being. However, you will be able to test using mods that have migrated already.

1.2: Migrate any old capability query

Thanks to the wrappers, it is very easy to migrate capability queries. You will find some examples below.

For block capabilities:

- IItemHandler handler = level.getCapability(Capabilities.ItemHandler.BLOCK, pos, side);
+ ResourceHandler<ItemResource> newHandler = level.getCapability(Capabilities.Item.BLOCK, pos, side);
+ IItemHandler handler = newHandler == null ? null : IItemHandler.of(newHandler);

  handler.insertItem(...);
  // ...

All the following code can remain unchanged for now.

For entity capabilities, this is very similar:

- IFluidHandler handler = entity.getCapability(Capabilities.FluidHandler.ENTITY, side);
+ ResourceHandler<FluidResource> newHandler = entity.getCapability(Capabilities.Fluid.ENTITY, side);
+ IFluidHandler handler = newHandler == null ? null : IFluidHandler.of(newHandler);

  handler.fill(...);
  // ...

For item capabilities, the migration is a bit more complicated because of the item contexts. For item handlers and energy storages, you can use ItemAccess.forStack which will apply modifications to the stack directly:

  ItemStack stack = ...;
- IEnergyStorage storage = stack.getCapability(Capabilities.EnergyStorage.ITEM);
+ EnergyHandler handler = ItemAccess.forStack(stack).getCapability(Capabilities.Energy.ITEM);
+ IEnergyStorage storage = handler == null ? null : IEnergyStorage.of(handler);

  storage.receive(...);
  // ...

For fluid handlers, the migration from IFluidHandlerItem is a bit trickier. It is simpler to use the FluidUtil.getFluidHandler(stack) method, which was updated to wrap a new handler into the old IFluidHandlerItem:

  ItemStack stack = ...;
- IFluidHandlerItem handler = stack.getCapability(Capabilities.FluidStorage.ITEM);
+ IFluidHandlerItem handler = FluidUtil.getFluidHandler(stack).orElse(null);

  handler.drain(...);
  // ...

1.3: Run and test your mod!

With these changes, your mod should compile and run again. Of course, your own containers will not be recognized because of step 1.1, but you can test your capability usage against vanilla containers and other methods that were updated. Some ideas:

  • For item transfer: chests, shulker box items, minecarts, …
  • For fluid transfer: cauldrons, buckets, …

Step 2: Migrating your capability providers

The next step is to migrate your capability provider implementations, such that you are able to register them to the new capabilities. In most cases, it should not be necessary to implement the new handler interfaces directly, since NeoForge provides many implementations of them. Additionally, most provided implementations in the old system (such as ItemStackHandler) have a new equivalent. The old provided implementations have been deprecated but not removed, and the new equivalent should be documented as @deprecated note in the javadoc.

Let’s look at an example. Say you registered a capability provider for a ComponentItemHandler. The javadoc of ComponentItemHandler hints at the required change:

/**
 * [...]
 * @deprecated Use {@link ItemAccessItemHandler} instead.
 */
@Deprecated(since = "1.21.9", forRemoval = true)
public class ComponentItemHandler implements IItemHandlerModifiable {
  // [...]
}

Indeed, ItemAccessItemHandler is the replacement. Capability registration can be updated as follows:

  void registerCapabilities(RegisterCapabilitiesEvent event) {
      // Old registration code:
-     event.registerItem(Capabilities.ItemHandler.ITEM, (stack, ctx) -> {
-         return new ComponentItemHandler(stack, DataComponents.CONTAINER, SLOT_COUNT);
-     }, MY_ITEM);
      // New registration code.
      // Notice the change in capability, that we have to take an itemAccess, and that we use the new helper:
+     event.registerItem(Capabilities.Item.ITEM, (stack, itemAccess) -> {
+         return new ItemAccessItemHandler(itemAccess, DataComponents.CONTAINER, SLOT_COUNT);
+     }, MY_ITEM);
  }

This step can hopefully be done incrementally for your mod. Once all the capability providers have been restored, your mod should be fully functional again. You can either stop there for the time being, or continue with step 3 to finish the migration.

Step 3: Moving away from the deprecated classes

Your mod is able to both query the new capabilities, as well as register providers for them. However, it is still full of classes that are deprecated for removal, especially if you used the strategy from step 1.1. Over time, you should look into migrating away from usage of the old capabilities, such that you are prepared for their removal in a future NeoForge update. The deprecated classes will be removed in a future NeoForge version cycle, i.e. NeoForge 21.10 or later. The exact removal version has not been decided yet.

Final words

I hope that the transition to this new API will not be too painful, and that the end result will make writing transfer code more intuitive and pleasant, as well as enable new original features (in particular, using transactions).

As usual, we are happy to provide help with the migration on Discord. It is also quite likely that there will be bugs in the new API given its size; should you find any, please don’t hesitate to let us know via GitHub. Requests for missing features are of course also welcome.

Acknowledgements

Thanks to CodexAdrian and Soaryn who started this effort over a year ago, before I took it over. Thanks to the NeoForge maintainers and everyone else from the community who reviewed this rework, tried it in their mod, or provided feedback in any other way. Special thanks to pupnewfster in particular who thoroughly reviewed most of my PRs. Finally, thanks to everyone who reviewed the Fabric Transfer API back in 2021, which paved the way for this rework.