Skip to content
GitLab
Explore
Sign in
Overview
Active
Stale
All
prefetch_fsdp_params_simple
88200493
·
prefetch params of previous fsdp module
·
Nov 19, 2021
fsdp_added_dummy_collective_calls_for_benchmarking
e8a510a0
·
fix
·
Nov 24, 2021
prefetch_fsdp_params_multiple_modules
899fe0cd
·
added option to disable double prefetch
·
Dec 02, 2021
anj/fix-lint
d3417ceb
·
FullyShardedDataParallel: only return full state dict on rank 0 (#885)
·
Jan 06, 2022
ngoyal_fix_for_multiple_backwards
40b9436f
·
dont remove post backward hooks for multiple backward fix
·
Feb 02, 2022
fixing_memory_issues_with_keeping_overlap
19155319
·
Get rid of warning
·
May 11, 2022
experts_lt_gpus_moe_reload_fix
2ec5c5e1
·
[fsdp][moe] reload checkpoint correctly for moe models with experts less than global world size
·
May 19, 2022
fixing_memory_issues_with_keeping_overlap_may24
91132c7e
·
Merge branch 'fixing_memory_issues_with_keeping_overlap' into...
·
Jun 22, 2022
ngoyal_bf16_changes
ba38cf32
·
bf16 changes and attribute for cpu activations
·
Jun 28, 2022
fsdp_ptd
592c0b93
·
update PG
·
Sep 27, 2022
ngoyal_8bit_all_gather
ff63d55e
·
fix
·
Feb 14, 2023
ngoyal_added_zero2_shard_modelparams_multiple_gpus
0b77de40
·
added option to do backward AG over smaller set of gpus instead of full DDP
·
May 20, 2023
fix_missing_params_in_unconsolidated_models
b5455b9b
·
Fix missing params in unconsolidated models
·
Sep 19, 2023
ngoyal_changes_for_pp
8e5c4162
·
changes for pp
·
Sep 22, 2023
ngoyal_changes_for_pp_fp8_fix_handle
45cd0385
·
split into wps fix P841842878 only and main_grad fix
·
Oct 01, 2023
ngoyal_changes_for_pp_fp8_fix_handle_grad_main
ad54660b
·
Add main_grad
·
Oct 02, 2023
ngoyal_changes_for_pp_fp8_event_sync
6b0e61e3
·
Changed to use event sync only
·
Oct 03, 2023
ngoyal_changes_for_pp_fp8_awgu
7aa7e9ba
·
Cleared backward hooks to avoid accumulating over iterations
·
Oct 09, 2023
add_main_grad_before_fwd
8cf28fa7
·
guard main_grad None
·
Oct 09, 2023
ngoyal_changes_for_pp_fp8_overlap
d7fe1136
·
Added `_run_all_post_backward_hooks`; removed stream sync if event sync
·
Oct 11, 2023
Prev
1
2
3
Next