This PR implements a core 'real' training loop in that it runs
deepseekv2 model using a number of Titan components to train on real
(C4) data with adamW and displays initial training loop metrics.

There is a lot more to be done but the goal here is to get a true
training loop going from which additional PRs will then improve upon it.

<img width="1192" alt="Screenshot 2025-05-29 at 7 41 01 PM"
src="https://github.com/user-attachments/assets/36ae2ff1-aa99-42c9-8b97-1e0a1ef8376e"
/>

A couple key highlights:
a - the model is now controllable via toml or cmd line just like Titan
main. Note that the expert parallel control is waiting for PR
#1244 to land...atm it just
manually puts ep to 2.
b - we use the HF deepseek tokenizer and as a result I had to make a
wrapper to deal with the bos and eos params passed by Titan.
c - loss metrics, tps, etc are displaying but MFU and tflops need to be
updated.

A lot more improvements will come shortly but for now want to land this
to ensure our base deepseek training loop is available to iterate on.

…used, not base class (pytorch#1244)

This PR:
Ensures that JobConfig extension values are included in the final merged
JobConfig, rather than just the custom fields and their values all
(incorrectly) set to zero.
Fix has gone from closure to a one line adjustment (credit @tianyu-l )
Unit testing has been increased to verify and catch any future breaks
like this (credit @jaysonfrancis )

Testing - verified all unit tests passing (with added assert for custom
field update) and expert parallel extension use case working.
Example:
~~~
@DataClass
class Parallelism:
    expert_parallel_degree: int = 2
    """ degree to parallelize experts """
~~~
results in 
~~~
parallelism=MergedParallelism(... expert_parallel_degree=0)
~~~
 which is not what is desired. 
 
 With PR:
 ~~~
parallelism=MergedParallelism(... expert_parallel_degree=2)
~~~

This PR implements a core 'real' training loop in that it runs
deepseekv2 model using a number of Titan components to train on real
(C4) data with adamW and displays initial training loop metrics.

There is a lot more to be done but the goal here is to get a true
training loop going from which additional PRs will then improve upon it.

<img width="1192" alt="Screenshot 2025-05-29 at 7 41 01 PM"
src="https://github.com/user-attachments/assets/36ae2ff1-aa99-42c9-8b97-1e0a1ef8376e"
/>

A couple key highlights:
a - the model is now controllable via toml or cmd line just like Titan
main. Note that the expert parallel control is waiting for PR
pytorch#1244 to land...atm it just
manually puts ep to 2.
b - we use the HF deepseek tokenizer and as a result I had to make a
wrapper to deal with the bos and eos params passed by Titan.
c - loss metrics, tps, etc are displaying but MFU and tflops need to be
updated.

A lot more improvements will come shortly but for now want to land this
to ensure our base deepseek training loop is available to iterate on.

…used, not base class (pytorch#1244)

This PR:
Ensures that JobConfig extension values are included in the final merged
JobConfig, rather than just the custom fields and their values all
(incorrectly) set to zero.
Fix has gone from closure to a one line adjustment (credit @tianyu-l )
Unit testing has been increased to verify and catch any future breaks
like this (credit @jaysonfrancis )

Testing - verified all unit tests passing (with added assert for custom
field update) and expert parallel extension use case working.
Example:
~~~
@DataClass
class Parallelism:
    expert_parallel_degree: int = 2
    """ degree to parallelize experts """
~~~
results in 
~~~
parallelism=MergedParallelism(... expert_parallel_degree=0)
~~~
 which is not what is desired. 
 
 With PR:
 ~~~
parallelism=MergedParallelism(... expert_parallel_degree=2)
~~~

This PR implements a core 'real' training loop in that it runs
deepseekv2 model using a number of Titan components to train on real
(C4) data with adamW and displays initial training loop metrics.

There is a lot more to be done but the goal here is to get a true
training loop going from which additional PRs will then improve upon it.

<img width="1192" alt="Screenshot 2025-05-29 at 7 41 01 PM"
src="https://github.com/user-attachments/assets/36ae2ff1-aa99-42c9-8b97-1e0a1ef8376e"
/>

A couple key highlights:
a - the model is now controllable via toml or cmd line just like Titan
main. Note that the expert parallel control is waiting for PR
pytorch#1244 to land...atm it just
manually puts ep to 2.
b - we use the HF deepseek tokenizer and as a result I had to make a
wrapper to deal with the bos and eos params passed by Titan.
c - loss metrics, tps, etc are displaying but MFU and tflops need to be
updated.

A lot more improvements will come shortly but for now want to land this
to ensure our base deepseek training loop is available to iterate on.

…used, not base class (pytorch#1244)

This PR:
Ensures that JobConfig extension values are included in the final merged
JobConfig, rather than just the custom fields and their values all
(incorrectly) set to zero.
Fix has gone from closure to a one line adjustment (credit @tianyu-l )
Unit testing has been increased to verify and catch any future breaks
like this (credit @jaysonfrancis )

Testing - verified all unit tests passing (with added assert for custom
field update) and expert parallel extension use case working.
Example:
~~~
@DataClass
class Parallelism:
    expert_parallel_degree: int = 2
    """ degree to parallelize experts """
~~~
results in 
~~~
parallelism=MergedParallelism(... expert_parallel_degree=0)
~~~
 which is not what is desired. 
 
 With PR:
 ~~~
parallelism=MergedParallelism(... expert_parallel_degree=2)
~~~

This PR implements a core 'real' training loop in that it runs
deepseekv2 model using a number of Titan components to train on real
(C4) data with adamW and displays initial training loop metrics.

There is a lot more to be done but the goal here is to get a true
training loop going from which additional PRs will then improve upon it.

<img width="1192" alt="Screenshot 2025-05-29 at 7 41 01 PM"
src="https://github.com/user-attachments/assets/36ae2ff1-aa99-42c9-8b97-1e0a1ef8376e"
/>

A couple key highlights:
a - the model is now controllable via toml or cmd line just like Titan
main. Note that the expert parallel control is waiting for PR
pytorch#1244 to land...atm it just
manually puts ep to 2.
b - we use the HF deepseek tokenizer and as a result I had to make a
wrapper to deal with the bos and eos params passed by Titan.
c - loss metrics, tps, etc are displaying but MFU and tflops need to be
updated.

A lot more improvements will come shortly but for now want to land this
to ensure our base deepseek training loop is available to iterate on.