What's Changed
* [CI, Doc] Update functorch source installation command by zou3519 in https://github.com/pytorch/rl/pull/446
* [BugFix] TransformedEnv attributes inheritance by vmoens in https://github.com/pytorch/rl/pull/467
* [Feature] Cleanup mocking envs init and new by vmoens in https://github.com/pytorch/rl/pull/469
* [Tests] Adding tensordict `__repr__` tests by sladebot in https://github.com/pytorch/rl/pull/435
* [Logging]: implement MLFlow logging integration by rayanht in https://github.com/pytorch/rl/pull/432
* [BugFix] MLFlow import fix by vmoens in https://github.com/pytorch/rl/pull/473
* [BugFix] Fixed pip install by brandonsj in https://github.com/pytorch/rl/pull/475
* [Features]: Changed `_inplace_update ` cls parameter passing in `__new__` by nicolas-dufour in https://github.com/pytorch/rl/pull/464
* [Feature]: ModelBased Envs by nicolas-dufour in https://github.com/pytorch/rl/pull/333
* [Feature] make ReplayBufferTrainer compatible with storing trajectories by vmoens in https://github.com/pytorch/rl/pull/476
* [Tutorial] DQN tutorial by vmoens in https://github.com/pytorch/rl/pull/474
* [Feature] reader hooks for GymLike by vmoens in https://github.com/pytorch/rl/pull/478
* [BugFix] TensorSpec.zero(None) failure fix by vmoens in https://github.com/pytorch/rl/pull/483
* [Feature]: Support for planners and CEM by nicolas-dufour in https://github.com/pytorch/rl/pull/384
* [Feature] Replaced `device_safe()` with `device` by ordinskiy in https://github.com/pytorch/rl/pull/485
* [Feature]: `TensorDictPrimer` transform by nicolas-dufour in https://github.com/pytorch/rl/pull/456
* [Feature]: `erase()` method for `torchrl.timeit` by nicolas-dufour in https://github.com/pytorch/rl/pull/480
* [Feature] Added support for single collector in sync_async_collector by nicolas-dufour in https://github.com/pytorch/rl/pull/482
* [BugFix] removing unwanted device_safe() by vmoens in https://github.com/pytorch/rl/pull/486
* [Refactoring] Refactored get_stats_random_rollout by nicolas-dufour in https://github.com/pytorch/rl/pull/481
* [Feature] VIP Integration by JasonMa2016 in https://github.com/pytorch/rl/pull/487
* [Refactoring] Minor tweaks to recorder and logger by nicolas-dufour in https://github.com/pytorch/rl/pull/489
* [Feature]: Deactivate typechecks in envs by nicolas-dufour in https://github.com/pytorch/rl/pull/490
* [BugFix] Vectorized td_lambda with gamma tensor does not match the serial version by vmoens in https://github.com/pytorch/rl/pull/400
* [BugFix] Fix TensorDictPrimer init by vmoens in https://github.com/pytorch/rl/pull/491
* [Feature] Optional auto-reset when done for collectors and batched envs by vmoens in https://github.com/pytorch/rl/pull/492
* [BugFix] Defaulting `passing_devices` to `None` by himjohntang in https://github.com/pytorch/rl/pull/477
* Revert "[BugFix] Defaulting `passing_devices` to `None`" by vmoens in https://github.com/pytorch/rl/pull/494
* [BugFix] Multi-agent fixes by vmoens in https://github.com/pytorch/rl/pull/488
* [BugFix] Defaulting `passing_devices` to `None` by vmoens in https://github.com/pytorch/rl/pull/495
* [Feature] Lazy initialization of CatTensors by vmoens in https://github.com/pytorch/rl/pull/497
* [Cleanup] Removing cuda 10.2 references by vmoens in https://github.com/pytorch/rl/pull/498
* [BugFix] Migration to pytorch org by vmoens in https://github.com/pytorch/rl/pull/499
* [Refactoring] Import at root to enable vmap monkey-patching by vmoens in https://github.com/pytorch/rl/pull/500
* [BugFix] python version for linting checks by vmoens in https://github.com/pytorch/rl/pull/502
* [Feature] Replay Buffers refactor by bamaxw in https://github.com/pytorch/rl/pull/330
* [Feature] Rename `step_tensordict` in `step_mdp` by romainjln in https://github.com/pytorch/rl/pull/512
* [Lint] re-instantiate F821 by vmoens in https://github.com/pytorch/rl/pull/516
* [BugFix] run_type_checks for TransformedEnvs by vmoens in https://github.com/pytorch/rl/pull/513
* [BugFix] making first_dim and last_dim negative in FlattenObservation when a parent is set by vmoens in https://github.com/pytorch/rl/pull/511
* [Feature] Add info dict key-spec pairs to observation_spec by tcbegley in https://github.com/pytorch/rl/pull/504
* [BugFix] Changing the dm_control import to fail if not installed by zeenolife in https://github.com/pytorch/rl/pull/515
* [CI] Add coverage with codecov by silvestrebahi in https://github.com/pytorch/rl/pull/523
* Revert "[CI] Add coverage with codecov" by vmoens in https://github.com/pytorch/rl/pull/525
* [Quality] Use relative imports for local c++ deps by apbard in https://github.com/pytorch/rl/pull/526
* [Feature] Nightly release by vmoens in https://github.com/pytorch/rl/pull/519
* [Feature] Add make_tensordict() function by sicong-huang in https://github.com/pytorch/rl/pull/522
* [Doc] Misc readme fixes by GavinPHR in https://github.com/pytorch/rl/pull/532
* [BugFix] Replacing inference_mode decorator with no_grad to fix state_dict loading error by GavinPHR in https://github.com/pytorch/rl/pull/530
* [BugFix] Transformed ParallelEnv meta data are broken when passing to device by vmoens in https://github.com/pytorch/rl/pull/531
* [Doc] Add coverage banner by vmoens in https://github.com/pytorch/rl/pull/533
* [BugFix] Fix colab link of coding_dqn.ipynb by Benjamin-eecs in https://github.com/pytorch/rl/pull/543
* [BugFix] Fix optional imports by vmoens in https://github.com/pytorch/rl/pull/535
* [BugFix] Restore missing keys in data collector output by tcbegley in https://github.com/pytorch/rl/pull/521
* [Lint] reorganize imports by apbard in https://github.com/pytorch/rl/pull/545
* [BugFix] Single-cpu compatibility by vmoens in https://github.com/pytorch/rl/pull/548
* [BugFix] vision install and other deps in optdeps by vmoens in https://github.com/pytorch/rl/pull/552
* [Feature] Implemented `device` argument for `modules.models` by yushiyangk in https://github.com/pytorch/rl/pull/524
* [BugFix] Fix ellipsis indexing of 2d TensorDicts by vmoens in https://github.com/pytorch/rl/pull/559
* [BugFix] Additive gaussian exploration spec fix by vmoens in https://github.com/pytorch/rl/pull/560
* [BugFix] Disabling video step for wandb by vmoens in https://github.com/pytorch/rl/pull/561
* [BugFix] Various device fix by vmoens in https://github.com/pytorch/rl/pull/558
* [Feature] Allow collectors to accept regular modules as policies by tcbegley in https://github.com/pytorch/rl/pull/546
* [BugFix] Fix push binary nightly action by psolikov in https://github.com/pytorch/rl/pull/566
* [BugFix] TensorDict comparison by vmoens in https://github.com/pytorch/rl/pull/567
* [BugFix] Fix SyncDataCollector reset by jrobine in https://github.com/pytorch/rl/pull/571
* [Doc] Banners on README.md by vmoens in https://github.com/pytorch/rl/pull/572
* [Feature] Log printing in alphabetical order when creating a replay buffer by nikhlrao in https://github.com/pytorch/rl/pull/573
* [BugFix] Add eps to reward normalization by vmoens in https://github.com/pytorch/rl/pull/574
* [BugFix] Fix argument for PPOLoss.get_entropy_bonus() by vmoens in https://github.com/pytorch/rl/pull/578
* [Feature] Restructure torchrl/objectives by sgrigory in https://github.com/pytorch/rl/pull/580
* [Docs] Documentation revamp by vmoens in https://github.com/pytorch/rl/pull/581
* [Doc] Publishing on pytorch.org by vmoens in https://github.com/pytorch/rl/pull/582
* Revert "[Doc] Publishing on pytorch.org" by vmoens in https://github.com/pytorch/rl/pull/584
* [Doc] Publishing on pytorch.org by vmoens in https://github.com/pytorch/rl/pull/585
* Revert "[Doc] Publishing on pytorch.org" by vmoens in https://github.com/pytorch/rl/pull/586
* [Doc] Publishing on pytorch.org by vmoens in https://github.com/pytorch/rl/pull/587
* [Feature] More restrictive tests on docstrings by vmoens in https://github.com/pytorch/rl/pull/457
* [BugFix] Wrong stack import in tests by vmoens in https://github.com/pytorch/rl/pull/590
* [Feature] Exclude `"_"` out_keys in tensordictmodel by jlesuffleur in https://github.com/pytorch/rl/pull/589
* [Feature]: Dreamer support by nicolas-dufour in https://github.com/pytorch/rl/pull/341
* [Doc] Missing doc for prototype RB by vmoens in https://github.com/pytorch/rl/pull/595
* [Feature] Update list of supported libraries by vmoens in https://github.com/pytorch/rl/pull/594
* [BugFix] Fix timeit count registration by vmoens in https://github.com/pytorch/rl/pull/598
* [Naming] Renaming `ProbabilisticTensorDictModule` keys by vmoens in https://github.com/pytorch/rl/pull/603
* [Feature] Categorical encoding for action space by artkorenev in https://github.com/pytorch/rl/pull/593
* [BugFix] ReplayBuffer's storage now signal back when changes happen by paulomarciano in https://github.com/pytorch/rl/pull/614
* [Doc] Typos in tensordict tutorial by PaLeroy in https://github.com/pytorch/rl/pull/621
* [Doc] Integrate knowledge base in docs by hatala91 in https://github.com/pytorch/rl/pull/622
* [Doc] Updating docs requirements by vmoens in https://github.com/pytorch/rl/pull/624
* [Feature] Make torchrl runnable without functorch and with gym==0.13 by vmoens in https://github.com/pytorch/rl/pull/386
* [Feature] Habitat integration by vmoens in https://github.com/pytorch/rl/pull/514
* [Feature] Checkpointing by vmoens in https://github.com/pytorch/rl/pull/549
* Add support for null `dim` argument in `TensorDict.squeeze` by jgonik in https://github.com/pytorch/rl/pull/608
* [Version] Updating to torch 1.13 by vmoens in https://github.com/pytorch/rl/pull/627
* [Feature] Sub-memmap tensors by vmoens in https://github.com/pytorch/rl/pull/626
* [BugFix] `copy_` changes the index if the dest and source memmap tensors share the same file location by vmoens in https://github.com/pytorch/rl/pull/631
* [Feature] Unfold transforms for folded `TransformedEnv` by alexanderlobov in https://github.com/pytorch/rl/pull/630
* [BugFix] make TensorDictReplayBuffer.extend call super().extend with stacked_td by vmoens in https://github.com/pytorch/rl/pull/634
* [BugFix] correct the use of `step_mdp` method in data collector by adityagandhamal in https://github.com/pytorch/rl/pull/637
* [Feature] Added `implement_for` decorator by ordinskiy in https://github.com/pytorch/rl/pull/618
* [Feature] Make DQN compatible with nn.Module by svarolgunes in https://github.com/pytorch/rl/pull/632
* [Example] Distributed Replay Buffer Prototype Example Implementation by adityagoel4512 in https://github.com/pytorch/rl/pull/615
* [Feature] Benchmark storage types by adityagoel4512 in https://github.com/pytorch/rl/pull/633
* [Feature] Remove wild imports in the library by sosmond in https://github.com/pytorch/rl/pull/642
* [BugFix] Prevent transform parent from being reassigned by jasonfkut in https://github.com/pytorch/rl/pull/641
* [Feature] Too many `deepcopy` in `transforms.py` by romainjln in https://github.com/pytorch/rl/pull/625
* [Naming] Rename keys_in to in_keys in transforms.py and related modules by sardaankita in https://github.com/pytorch/rl/pull/656
* [Refactoring] Refactor dreamer helper in smaller pieces by vmoens in https://github.com/pytorch/rl/pull/662
* [Feature] VIPRewardTransform by vmoens in https://github.com/pytorch/rl/pull/658
* [BugFix] make_trainer possible bug for on-policy cases by albertbou92 in https://github.com/pytorch/rl/pull/655
* [Naming] Fixing key names by vmoens in https://github.com/pytorch/rl/pull/668
* [Test] Check dtypes of envs by vmoens in https://github.com/pytorch/rl/pull/666
* [Refactor] Relying on the standalone tensordict -- phase 1 by vmoens in https://github.com/pytorch/rl/pull/650
* [Doc] More doc on trainers by vmoens in https://github.com/pytorch/rl/pull/663
* [BugFix] PPO example GAE import by albertbou92 in https://github.com/pytorch/rl/pull/671
* [BugFix] Use GitHub for flake8 pre-commit hook by vmoens in https://github.com/pytorch/rl/pull/679
* [BugFix] Update to strict select by vmoens in https://github.com/pytorch/rl/pull/675
* [Feature] Auto-compute stats for ObservationNorm by romainjln in https://github.com/pytorch/rl/pull/669
* [Doc] _make_collector helper function by albertbou92 in https://github.com/pytorch/rl/pull/678
* [Doc] BatchSubSampler class docstrings example by albertbou92 in https://github.com/pytorch/rl/pull/677
* [BugFix] PPO objective crashes if advantage_module is None by albertbou92 in https://github.com/pytorch/rl/pull/676
* [Refactor] Refactor 'next_' into nested tensordicts by vmoens in https://github.com/pytorch/rl/pull/649
* [Doc] More doc about environments by vmoens in https://github.com/pytorch/rl/pull/683
* [Doc] Fix missing tensordict install for doc by vmoens in https://github.com/pytorch/rl/pull/685
* [CI] Added CircleCI pipeline to test compatibility across supported gym versions by ordinskiy in https://github.com/pytorch/rl/pull/645
* [BugFix] ConvNet forward method with tensors of more than 4 dimensions by albertbou92 in https://github.com/pytorch/rl/pull/686
* [Feature] add `standard_normal` for RewardScaling by adityagandhamal in https://github.com/pytorch/rl/pull/682
* [Feature] Jumanji envs by yingchenlin in https://github.com/pytorch/rl/pull/674
* [Feature] Default collate_fn by vmoens in https://github.com/pytorch/rl/pull/688
* [BugFix] Fix Examples by vmoens in https://github.com/pytorch/rl/pull/687
* [Refactoring] Replace direct gym version checks with decorated functions () by ordinskiy in https://github.com/pytorch/rl/pull/691
* Version 0.0.3 by vmoens in https://github.com/pytorch/rl/pull/696
* [Docs] Host TensorDict docs inside TorchRL docs by tcbegley in https://github.com/pytorch/rl/pull/693
* [BugFix] Fix docs build by tcbegley in https://github.com/pytorch/rl/pull/698
* [BugFix] Proper error messages for orphan transform creation by vmoens in https://github.com/pytorch/rl/pull/697
* [Feature] Append, init and insert transforms in ReplayBuffer by altre in https://github.com/pytorch/rl/pull/695
* [Feature] A2C objective class and train example by albertbou92 in https://github.com/pytorch/rl/pull/680
* [Doc, Test] Add A2C script test and doc by vmoens in https://github.com/pytorch/rl/pull/702
* [BugFix] Initialising the classes LazyTensorStorage with a nested TensorDict raises error by albertbou92 in https://github.com/pytorch/rl/pull/703
* [BugFix] Fix init_random_frames in A2C example test by vmoens in https://github.com/pytorch/rl/pull/706
* [Formatting] Upgrade formatting libs by vmoens in https://github.com/pytorch/rl/pull/705
* [Doc] Document undefined symbol error with torch version < 1.13 by nickspell in https://github.com/pytorch/rl/pull/707
* [Doc] Tuto integration by vmoens in https://github.com/pytorch/rl/pull/681
* [Quality] Deprecate .ipynb tutos by vmoens in https://github.com/pytorch/rl/pull/710
* [Test] Fix wrong skip message when functorch is installed by vmoens in https://github.com/pytorch/rl/pull/711
* [BugFix, Doc] Clone TensorDict docs into _local_build by tcbegley in https://github.com/pytorch/rl/pull/712
* [Feature] Migrate to `tensordict.nn.TensorDictModule` by tcbegley in https://github.com/pytorch/rl/pull/700
* [Doc] Fix Tutos TODOs by vmoens in https://github.com/pytorch/rl/pull/713
* [BugFix] RoundRobinWriter, possible duplicated code in the extend method by albertbou92 in https://github.com/pytorch/rl/pull/709
* [Feature] Add OptimizerHook by aakhundov in https://github.com/pytorch/rl/pull/716
* [Feature] Support for in-place functionalization by tcbegley in https://github.com/pytorch/rl/pull/714
* [BugFix] Fix TorchRL demo tutorial by vmoens in https://github.com/pytorch/rl/pull/721
* [Docs] Update tutorial links in readme by tcbegley in https://github.com/pytorch/rl/pull/724
* [Feature] Extend PPO loss helper to allow for more customisation by albertbou92 in https://github.com/pytorch/rl/pull/718
* [BugFix] Model maker functions for A2C and PPO fail for discrete action space envs by albertbou92 in https://github.com/pytorch/rl/pull/717
* [Minor] docstrings and setup fixes by vmoens in https://github.com/pytorch/rl/pull/726
* [BugFix] Avoid wrongfully erasing observation keys from specs in CatTensors by vmoens in https://github.com/pytorch/rl/pull/727
* [BugFix] Avoid wrongfully erasing observation keys from tensordict in CatTensors by vmoens in https://github.com/pytorch/rl/pull/729
* [Doc] More doc for data collectors by vmoens in https://github.com/pytorch/rl/pull/732
* [Feature] Port test_fake_tensordict to torchrl by vmoens in https://github.com/pytorch/rl/pull/731
* [Feature] Use ObservationNorm.init_stats for stats computation in example scripts by romainjln in https://github.com/pytorch/rl/pull/715
* [BugFix] init_stats over multiple dimensions by vmoens in https://github.com/pytorch/rl/pull/735
* [Refactor] logger creation in examples by acforvs in https://github.com/pytorch/rl/pull/733
* [Feature] Brax envs by yingchenlin in https://github.com/pytorch/rl/pull/722
* [Refactor] Adopt prototype ProbabilisticTensorDictModule and ProbabilisticTensorDictSequential by tcbegley in https://github.com/pytorch/rl/pull/728
* [Doc] Link to doc in README by vmoens in https://github.com/pytorch/rl/pull/740
* [Feature] Make GAE return a 'value_target' entry by vmoens in https://github.com/pytorch/rl/pull/741
* [Feature] SamplerWithoutReplacement by vmoens in https://github.com/pytorch/rl/pull/742
* [Doc, CI] Update doc workflow to run on PR and only publishes doc on main. by EmGarr in https://github.com/pytorch/rl/pull/745
* [Feature] Better advantage API for higher order derivatives by vmoens in https://github.com/pytorch/rl/pull/744
* [Refactor] Cosmetic improvements to advantage modules by vmoens in https://github.com/pytorch/rl/pull/746
* [BugFix] Fix NoopReset in parallel settings by vmoens in https://github.com/pytorch/rl/pull/747
* [Refactor] Remove env.is_done attribute by vmoens in https://github.com/pytorch/rl/pull/748
* [Refactor] Drop prototype imports by tcbegley in https://github.com/pytorch/rl/pull/738
* [BugFix] Fixes for `speed` branch merge on tensordict by vmoens in https://github.com/pytorch/rl/pull/755
* [BugFix] Fix size-match unsqueeze deprecation by vmoens in https://github.com/pytorch/rl/pull/750
* [Feature] FrameSkipTransform by vmoens in https://github.com/pytorch/rl/pull/749
* [BugFix] Better memory management for collectors by vmoens in https://github.com/pytorch/rl/pull/763
* Minor cleaning in BaseEnv classes by matteobettini in https://github.com/pytorch/rl/pull/767
* Revert "Minor cleaning in BaseEnv classes" by vmoens in https://github.com/pytorch/rl/pull/768
* Cleaning in envs `common.py` by matteobettini in https://github.com/pytorch/rl/pull/769
* Making `_set_seed` abstract by matteobettini in https://github.com/pytorch/rl/pull/770
* [Feature] Remove the Nd*TensorSpec classes by riiswa in https://github.com/pytorch/rl/pull/772
* [BugFix] Reinstantiate custom value key for multioutput value networks by vmoens in https://github.com/pytorch/rl/pull/754
* [Feature] Add Step Counter transform by riiswa in https://github.com/pytorch/rl/pull/756
* [BugFix] Batched environments with non empty batch size by matteobettini in https://github.com/pytorch/rl/pull/774
* Allow undounded boxes creation from gym spaces by matteobettini in https://github.com/pytorch/rl/pull/778
* [BugFix] Doc built cmake error by vmoens in https://github.com/pytorch/rl/pull/780
* [Feature] Lazy TensorClass storage by tcbegley in https://github.com/pytorch/rl/pull/752
* [BugFix] SyncDataCollector init when device and env_device are different by albertbou92 in https://github.com/pytorch/rl/pull/765
* [Feature] RewardSum transform by albertbou92 in https://github.com/pytorch/rl/pull/751
* [BugFix] Fix PPO clip by vmoens in https://github.com/pytorch/rl/pull/786
* [Feature] MultiDiscreteTensorSpec by riiswa in https://github.com/pytorch/rl/pull/783
* [Doc] Doc revamp by vmoens in https://github.com/pytorch/rl/pull/782
* [BugFix] `ParallelEnv` handling of done flag by matteobettini in https://github.com/pytorch/rl/pull/788
* [BugFix] Sorting nested keys by matteobettini in https://github.com/pytorch/rl/pull/787
* [Doc] README index by vmoens in https://github.com/pytorch/rl/pull/791
* Add windows wheel build to CircleCI by yohann-benchetrit in https://github.com/pytorch/rl/pull/759
* [Algorithm] MPPI planner by vmoens in https://github.com/pytorch/rl/pull/701
* [Doc] Better doc links by vmoens in https://github.com/pytorch/rl/pull/795
* [Doc] Missing headers by vmoens in https://github.com/pytorch/rl/pull/796
* [Doc] Knowledge base section by vmoens in https://github.com/pytorch/rl/pull/797
* [Feature] Vmas library wrapper by matteobettini in https://github.com/pytorch/rl/pull/785
* [Doc] Duplicate `HabitatEnv` entry in docs by matteobettini in https://github.com/pytorch/rl/pull/798
* [Feature] MultiDiscreteTensorSpec nvec with several axes by riiswa in https://github.com/pytorch/rl/pull/789
* [Refactor] Graduate Replay Buffer prototype by KamilPiechowiak in https://github.com/pytorch/rl/pull/794
* [BugFix] Solve R3MTransform init problem by vmoens in https://github.com/pytorch/rl/pull/803
* [Refactor] Simplify FlattenObservation default kwargs by vmoens in https://github.com/pytorch/rl/pull/805
* [Format] Fix lint by vmoens in https://github.com/pytorch/rl/pull/811
* [Doc, BugFix] Fix tutos errors by vmoens in https://github.com/pytorch/rl/pull/817
* [Doc] Pretrained models tutorial by vmoens in https://github.com/pytorch/rl/pull/814
* [Doc, BugFix] Fix tensordictmodule tutorial by vmoens in https://github.com/pytorch/rl/pull/819
* [BugFix] Fix MultOneHotDiscreteTensorSpec.is_in by riiswa in https://github.com/pytorch/rl/pull/818
* [Doc] Using R3M with a replay buffer by vmoens in https://github.com/pytorch/rl/pull/820
* [CodeQuality] call all() without making a list by riiswa in https://github.com/pytorch/rl/pull/821
* [BugFix] [Feature] "_reset" flag for env reset by matteobettini in https://github.com/pytorch/rl/pull/800
* [CI] Add unit test workflows for Windows by yohann-benchetrit in https://github.com/pytorch/rl/pull/804
* [BugFix] Fix habitat integration and doc by vmoens in https://github.com/pytorch/rl/pull/812
* [Minor] Better error reporting by vmoens in https://github.com/pytorch/rl/pull/822
* [Minor] Add ninja to deps in toml file by vmoens in https://github.com/pytorch/rl/pull/823
* [BugFix] Device of info specs by vmoens in https://github.com/pytorch/rl/pull/824
* [BugFix] Fix envs specs and info reading by vmoens in https://github.com/pytorch/rl/pull/825
* [Feature] Dtype in vmas tests by matteobettini in https://github.com/pytorch/rl/pull/827
* [BugFix] Fix R3M observation spec transform by vmoens in https://github.com/pytorch/rl/pull/830
* small change to make robandpdx a contributor by robandpdx in https://github.com/pytorch/rl/pull/831
* [Feature] Exclude and select transforms by vmoens in https://github.com/pytorch/rl/pull/832
* [BugFix] Updating Recorder to accomodate "solved" key by ShahRutav in https://github.com/pytorch/rl/pull/833
* [BugFIx] Changed "set_count" set in collectors by matteobettini in https://github.com/pytorch/rl/pull/835
* [Algorithm] Td3 by BY571 in https://github.com/pytorch/rl/pull/684
* [Doc] A Succinct Summary of Reinforcement Learning by vmoens in https://github.com/pytorch/rl/pull/840
* [Feature, BugFix] ObservationNorm keep_dims and RewardSum init by vmoens in https://github.com/pytorch/rl/pull/839
* [BugFix] Improve done checking of collectors by matteobettini in https://github.com/pytorch/rl/pull/838
* [BugFix] Sync with tensordict (meta-tensor deprecation) by vmoens in https://github.com/pytorch/rl/pull/842
* [Feature] Refactor CatFrames using a proper preallocated buffer by vmoens in https://github.com/pytorch/rl/pull/847
* [CI] Add Github-Actions workflows for Windows wheels & nightly-build by yohann-benchetrit in https://github.com/pytorch/rl/pull/837
* [Doc] Fix broken link Dreamer by atonkamanda in https://github.com/pytorch/rl/pull/853
* [BugFix] Loading state_dict on uninitialized CatFrames by vmoens in https://github.com/pytorch/rl/pull/855
* [Refactor] Move loggers to `torchrl.record` by vmoens in https://github.com/pytorch/rl/pull/854
* [Refactor] specs batch size refactoring by vmoens in https://github.com/pytorch/rl/pull/829
* [Feature] Max pool Transform by albertbou92 in https://github.com/pytorch/rl/pull/841
* [Feature] Refactor advantages for continuous batches by vmoens in https://github.com/pytorch/rl/pull/848
* [BugFix, Doc] Minor fix in doc by vmoens in https://github.com/pytorch/rl/pull/858
* [Versioning] Version 0.0.4a by vmoens in https://github.com/pytorch/rl/pull/859
* [Feature] Vmas to device by matteobettini in https://github.com/pytorch/rl/pull/850
* [BugFix] Fix zero-ing from specs in RewardSum by vmoens in https://github.com/pytorch/rl/pull/860
* [Feature] Loading R3M and VIP from ResNet by vmoens in https://github.com/pytorch/rl/pull/863
* [Feature] SAC V2 by vmoens in https://github.com/pytorch/rl/pull/864
* [BugFix] Avoid collision of "step_count" key from transform and collector by vmoens in https://github.com/pytorch/rl/pull/868
* [Refactor] Better init for CatFrames buffers + removing default init values by vmoens in https://github.com/pytorch/rl/pull/874
* [Refactor] Minor refactorings to envs by vmoens in https://github.com/pytorch/rl/pull/872
* [Refactor] Removing inplace transform attribute by vmoens in https://github.com/pytorch/rl/pull/871
* [BugFix] Run checks when creating fake_td by vmoens in https://github.com/pytorch/rl/pull/877
* [Refactor] Box device by vmoens in https://github.com/pytorch/rl/pull/881
* [Feature] Multithreaded env by sgrigory in https://github.com/pytorch/rl/pull/734
* [Refactor] Turn off default advantage normalization in PPO by vmoens in https://github.com/pytorch/rl/pull/887
* [CI] Fix habitat-gym imports by vmoens in https://github.com/pytorch/rl/pull/890
* [CI] Fix cuda versions by vmoens in https://github.com/pytorch/rl/pull/889
* [CI] Fix windows install by vmoens in https://github.com/pytorch/rl/pull/888
* MacOS CPU unit test workflow using GitHub Actions by robandpdx in https://github.com/pytorch/rl/pull/886
* Linux CPU unit test workflow using GitHub Actions by robandpdx in https://github.com/pytorch/rl/pull/826
* [Major, BugFix, Test] Refactor Transforms tests by vmoens in https://github.com/pytorch/rl/pull/878
* [Bugfix] Codecov does not cover multiprocessed tests 879 by kadeng in https://github.com/pytorch/rl/pull/893
* [CI, BugFix] Fix gym related errors by vmoens in https://github.com/pytorch/rl/pull/895
* [WIP] Linux GPU unit test workflow using GitHub Actions by robandpdx in https://github.com/pytorch/rl/pull/885
* [BugFix] Compose cloning fix by vmoens in https://github.com/pytorch/rl/pull/899
* [Feature] Simplifying collector envs by vmoens in https://github.com/pytorch/rl/pull/870
* [CI,Feature] Upgrade to gymnasium by vmoens in https://github.com/pytorch/rl/pull/898
* [Doc] Add record utils to doc by vmoens in https://github.com/pytorch/rl/pull/904
* [Test] Improve exception message match by apbard in https://github.com/pytorch/rl/pull/906
* [BugFix] Dreamer helpers are broken with batched envs by vmoens in https://github.com/pytorch/rl/pull/903
* [Feature] RandomCropTensorDict transform by vmoens in https://github.com/pytorch/rl/pull/908
* [Versioning] Version 0.0.4b by vmoens in https://github.com/pytorch/rl/pull/909
New Contributors
* sladebot made their first contribution in https://github.com/pytorch/rl/pull/435
* rayanht made their first contribution in https://github.com/pytorch/rl/pull/432
* brandonsj made their first contribution in https://github.com/pytorch/rl/pull/475
* ordinskiy made their first contribution in https://github.com/pytorch/rl/pull/485
* JasonMa2016 made their first contribution in https://github.com/pytorch/rl/pull/487
* himjohntang made their first contribution in https://github.com/pytorch/rl/pull/477
* romainjln made their first contribution in https://github.com/pytorch/rl/pull/512
* apbard made their first contribution in https://github.com/pytorch/rl/pull/526
* sicong-huang made their first contribution in https://github.com/pytorch/rl/pull/522
* psolikov made their first contribution in https://github.com/pytorch/rl/pull/566
* jrobine made their first contribution in https://github.com/pytorch/rl/pull/571
* nikhlrao made their first contribution in https://github.com/pytorch/rl/pull/573
* sgrigory made their first contribution in https://github.com/pytorch/rl/pull/580
* jlesuffleur made their first contribution in https://github.com/pytorch/rl/pull/589
* artkorenev made their first contribution in https://github.com/pytorch/rl/pull/593
* paulomarciano made their first contribution in https://github.com/pytorch/rl/pull/614
* hatala91 made their first contribution in https://github.com/pytorch/rl/pull/622
* jgonik made their first contribution in https://github.com/pytorch/rl/pull/608
* adityagandhamal made their first contribution in https://github.com/pytorch/rl/pull/637
* svarolgunes made their first contribution in https://github.com/pytorch/rl/pull/632
* adityagoel4512 made their first contribution in https://github.com/pytorch/rl/pull/615
* jasonfkut made their first contribution in https://github.com/pytorch/rl/pull/641
* sardaankita made their first contribution in https://github.com/pytorch/rl/pull/656
* albertbou92 made their first contribution in https://github.com/pytorch/rl/pull/655
* yingchenlin made their first contribution in https://github.com/pytorch/rl/pull/674
* altre made their first contribution in https://github.com/pytorch/rl/pull/695
* nickspell made their first contribution in https://github.com/pytorch/rl/pull/707
* aakhundov made their first contribution in https://github.com/pytorch/rl/pull/716
* acforvs made their first contribution in https://github.com/pytorch/rl/pull/733
* EmGarr made their first contribution in https://github.com/pytorch/rl/pull/745
* matteobettini made their first contribution in https://github.com/pytorch/rl/pull/767
* riiswa made their first contribution in https://github.com/pytorch/rl/pull/772
* yohann-benchetrit made their first contribution in https://github.com/pytorch/rl/pull/759
* KamilPiechowiak made their first contribution in https://github.com/pytorch/rl/pull/794
* robandpdx made their first contribution in https://github.com/pytorch/rl/pull/831
* ShahRutav made their first contribution in https://github.com/pytorch/rl/pull/833
* BY571 made their first contribution in https://github.com/pytorch/rl/pull/684
* atonkamanda made their first contribution in https://github.com/pytorch/rl/pull/853
* kadeng made their first contribution in https://github.com/pytorch/rl/pull/893
**Full Changelog**: https://github.com/pytorch/rl/compare/v0.0.2a...v0.0.4b