K33w3/Project2.1

Reward System Updates -> Adjust the reward system for both the Crawler and Push Block environments. Add complex terrain to the Crawler environment, ensuring proper rewards for navigation.

Graph and Episode Length Analysis Analyze the episode length and all generated graphs to identify trends, anomalies, and areas for improvement. Create a detailed comparison of the performance metrics between different configurations. Propose adjustments to enhance model performance based on insights.

Train Environments with Updated Parameters Modify .yaml parameters for both environments based on the initial analysis. Train the environments with these new configurations, ensuring thorough logging of episode length, rewards, and other key metrics. Save and backup all training results for future comparisons.

Develop Grid Search Algorithm for YAML Optimization Design and implement a grid search algorithm to identify the optimal configurations in .yaml files for ML-Agents. Apply the algorithm to both environments and document the outcomes. Include logging and visualization of the search results for easier analysis.

Research which algorithm should work better in each of the environments. Decide which metrics should be put as the most crucial ones and why they are there...

Performance Metrics -> Define research questions for this phase. Compile a list of performance metrics for evaluation (e.g., training time, CPU/GPU usage, framerate). Document experiments and ensure reproducibility using configuration files.

Finish the report draft. Leave conclusion and discussion out for the data.

Begin training in the "Push Block" environment using the default parameters. Monitor the TensorBoard graphs to analyze the average number of steps required for effective training.

Environment Setup -> Set up the Crawler Environment. Set up the Push Block Environment. Validate that environments load correctly and clean up unnecessary elements.

Finish the report draft. Leave conclusion and discussion out for the data.

Define new training parameters optimized for conducting a detailed comparison, ensuring they highlight differences in performance and outcomes effectively.

Begin training in the "Push Block" environment using the default parameters. Monitor the TensorBoard graphs to analyze the average number of steps required for effective training.

Begin training in the "Crawler" environment using the default parameters. Monitor the TensorBoard graphs to analyze the average number of steps required for effective training.

Report and Presentation -> Draft the main report structure, excluding metrics and graphs. Develop a basic presentation template with placeholders for visuals. Compile the final report and presentation with complete data and references.

Research PPO (Proximal Policy Optimization) -> Conduct a detailed analysis of the PPO algorithm, its architecture, and applications in ML-Agents. Focus on the following aspects: Overview of PPO and its advantages/disadvantages. Key hyperparameters (e.g., learning rate, batch size, clip range). PPO's performance in Unity ML-Agents environments. Potential challenges and best practices for tuning PPO.

Research SAC (Soft Actor-Critic) -> Conduct a detailed analysis of the SAC algorithm, its architecture, and applications in ML-Agents. Focus on the following aspects: Overview of SAC and its advantages/disadvantages. Key hyperparameters (e.g., entropy coefficient, learning rate, replay buffer size). SAC's performance in Unity ML-Agents environments. Potential challenges and best practices for tuning SAC.

Environment Setup -> Set up the Crawler Environment. Set up the Push Block Environment. Validate that environments load correctly and clean up unnecessary elements.

Reward System Updates -> Adjust the reward system for both the Crawler and Push Block environments. Add complex terrain to the Crawler environment, ensuring proper rewards for navigation.

Report and Presentation -> Draft the main report structure, excluding metrics and graphs. Develop a basic presentation template with placeholders for visuals. Compile the final report and presentation with complete data and references.

Performance Metrics -> Define research questions for this phase. Compile a list of performance metrics for evaluation (e.g., training time, CPU/GPU usage, framerate). Document experiments and ensure reproducibility using configuration files.

Research which algorithm should work better in each of the environments. Decide which metrics should be put as the most crucial ones and why they are there...

Research PPO (Proximal Policy Optimization) -> Conduct a detailed analysis of the PPO algorithm, its architecture, and applications in ML-Agents. Focus on the following aspects: Overview of PPO and its advantages/disadvantages. Key hyperparameters (e.g., learning rate, batch size, clip range). PPO's performance in Unity ML-Agents environments. Potential challenges and best practices for tuning PPO.

Research SAC (Soft Actor-Critic) -> Conduct a detailed analysis of the SAC algorithm, its architecture, and applications in ML-Agents. Focus on the following aspects: Overview of SAC and its advantages/disadvantages. Key hyperparameters (e.g., entropy coefficient, learning rate, replay buffer size). SAC's performance in Unity ML-Agents environments. Potential challenges and best practices for tuning SAC.