sf_log.txt

[2025-01-19 23:58:15,927][00404] Saving configuration to /content/train_dir/default_experiment/config.json...
[2025-01-19 23:58:15,930][00404] Rollout worker 0 uses device cpu
[2025-01-19 23:58:15,931][00404] Rollout worker 1 uses device cpu
[2025-01-19 23:58:15,933][00404] Rollout worker 2 uses device cpu
[2025-01-19 23:58:15,934][00404] Rollout worker 3 uses device cpu
[2025-01-19 23:58:15,936][00404] Rollout worker 4 uses device cpu
[2025-01-19 23:58:15,937][00404] Rollout worker 5 uses device cpu
[2025-01-19 23:58:15,939][00404] Rollout worker 6 uses device cpu
[2025-01-19 23:58:15,940][00404] Rollout worker 7 uses device cpu
[2025-01-19 23:58:16,106][00404] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2025-01-19 23:58:16,108][00404] InferenceWorker_p0-w0: min num requests: 2
[2025-01-19 23:58:16,145][00404] Starting all processes...
[2025-01-19 23:58:16,147][00404] Starting process learner_proc0
[2025-01-19 23:58:16,190][00404] Starting all processes...
[2025-01-19 23:58:16,198][00404] Starting process inference_proc0-0
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc0
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc1
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc2
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc3
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc4
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc5
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc6
[2025-01-19 23:58:16,200][00404] Starting process rollout_proc7
[2025-01-19 23:58:30,990][02242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2025-01-19 23:58:30,994][02242] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2025-01-19 23:58:31,049][02242] Num visible devices: 1
[2025-01-19 23:58:31,085][02242] Starting seed is not provided
[2025-01-19 23:58:31,087][02242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2025-01-19 23:58:31,088][02242] Initializing actor-critic model on device cuda:0
[2025-01-19 23:58:31,089][02242] RunningMeanStd input shape: (3, 72, 128)
[2025-01-19 23:58:31,092][02242] RunningMeanStd input shape: (1,)
[2025-01-19 23:58:31,178][02242] ConvEncoder: input_channels=3
[2025-01-19 23:58:31,235][02257] Worker 1 uses CPU cores [1]
[2025-01-19 23:58:31,330][02258] Worker 2 uses CPU cores [0]
[2025-01-19 23:58:31,416][02261] Worker 5 uses CPU cores [1]
[2025-01-19 23:58:31,534][02260] Worker 4 uses CPU cores [0]
[2025-01-19 23:58:31,585][02262] Worker 6 uses CPU cores [0]
[2025-01-19 23:58:31,631][02263] Worker 7 uses CPU cores [1]
[2025-01-19 23:58:31,717][02256] Worker 0 uses CPU cores [0]
[2025-01-19 23:58:31,720][02255] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2025-01-19 23:58:31,720][02255] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2025-01-19 23:58:31,740][02255] Num visible devices: 1
[2025-01-19 23:58:31,747][02259] Worker 3 uses CPU cores [1]
[2025-01-19 23:58:31,753][02242] Conv encoder output size: 512
[2025-01-19 23:58:31,753][02242] Policy head output size: 512
[2025-01-19 23:58:31,804][02242] Created Actor Critic model with architecture:
[2025-01-19 23:58:31,804][02242] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ELU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ELU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2025-01-19 23:58:32,173][02242] Using optimizer <class 'torch.optim.adam.Adam'>
[2025-01-19 23:58:36,100][00404] Heartbeat connected on Batcher_0
[2025-01-19 23:58:36,107][00404] Heartbeat connected on InferenceWorker_p0-w0
[2025-01-19 23:58:36,116][00404] Heartbeat connected on RolloutWorker_w0
[2025-01-19 23:58:36,122][00404] Heartbeat connected on RolloutWorker_w1
[2025-01-19 23:58:36,125][00404] Heartbeat connected on RolloutWorker_w2
[2025-01-19 23:58:36,128][00404] Heartbeat connected on RolloutWorker_w3
[2025-01-19 23:58:36,133][00404] Heartbeat connected on RolloutWorker_w4
[2025-01-19 23:58:36,136][00404] Heartbeat connected on RolloutWorker_w5
[2025-01-19 23:58:36,142][00404] Heartbeat connected on RolloutWorker_w6
[2025-01-19 23:58:36,147][00404] Heartbeat connected on RolloutWorker_w7
[2025-01-19 23:58:37,565][02242] No checkpoints found
[2025-01-19 23:58:37,565][02242] Did not load from checkpoint, starting from scratch!
[2025-01-19 23:58:37,567][02242] Initialized policy 0 weights for model version 0
[2025-01-19 23:58:37,569][02242] LearnerWorker_p0 finished initialization!
[2025-01-19 23:58:37,572][02242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2025-01-19 23:58:37,570][00404] Heartbeat connected on LearnerWorker_p0
[2025-01-19 23:58:37,772][02255] RunningMeanStd input shape: (3, 72, 128)
[2025-01-19 23:58:37,773][02255] RunningMeanStd input shape: (1,)
[2025-01-19 23:58:37,784][02255] ConvEncoder: input_channels=3
[2025-01-19 23:58:37,882][02255] Conv encoder output size: 512
[2025-01-19 23:58:37,882][02255] Policy head output size: 512
[2025-01-19 23:58:37,916][00404] Inference worker 0-0 is ready!
[2025-01-19 23:58:37,917][00404] All inference workers are ready! Signal rollout workers to start!
[2025-01-19 23:58:38,117][02263] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,119][02259] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,114][02257] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,123][02261] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,139][02262] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,143][02258] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,140][02256] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,138][02260] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-19 23:58:38,833][02256] Decorrelating experience for 0 frames...
[2025-01-19 23:58:39,217][02256] Decorrelating experience for 32 frames...
[2025-01-19 23:58:39,428][02259] Decorrelating experience for 0 frames...
[2025-01-19 23:58:39,431][02261] Decorrelating experience for 0 frames...
[2025-01-19 23:58:39,434][02263] Decorrelating experience for 0 frames...
[2025-01-19 23:58:40,003][02256] Decorrelating experience for 64 frames...
[2025-01-19 23:58:40,171][02261] Decorrelating experience for 32 frames...
[2025-01-19 23:58:40,237][02262] Decorrelating experience for 0 frames...
[2025-01-19 23:58:40,280][02257] Decorrelating experience for 0 frames...
[2025-01-19 23:58:40,808][02256] Decorrelating experience for 96 frames...
[2025-01-19 23:58:41,278][02259] Decorrelating experience for 32 frames...
[2025-01-19 23:58:41,277][02260] Decorrelating experience for 0 frames...
[2025-01-19 23:58:41,530][02263] Decorrelating experience for 32 frames...
[2025-01-19 23:58:41,617][00404] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2025-01-19 23:58:41,619][02257] Decorrelating experience for 32 frames...
[2025-01-19 23:58:42,204][02262] Decorrelating experience for 32 frames...
[2025-01-19 23:58:42,705][02260] Decorrelating experience for 32 frames...
[2025-01-19 23:58:42,731][02258] Decorrelating experience for 0 frames...
[2025-01-19 23:58:43,346][02261] Decorrelating experience for 64 frames...
[2025-01-19 23:58:43,770][02259] Decorrelating experience for 64 frames...
[2025-01-19 23:58:44,034][02257] Decorrelating experience for 64 frames...
[2025-01-19 23:58:44,325][02263] Decorrelating experience for 64 frames...
[2025-01-19 23:58:44,371][02262] Decorrelating experience for 64 frames...
[2025-01-19 23:58:44,487][02258] Decorrelating experience for 32 frames...
[2025-01-19 23:58:44,982][02260] Decorrelating experience for 64 frames...
[2025-01-19 23:58:45,061][02259] Decorrelating experience for 96 frames...
[2025-01-19 23:58:45,431][02257] Decorrelating experience for 96 frames...
[2025-01-19 23:58:45,957][02263] Decorrelating experience for 96 frames...
[2025-01-19 23:58:46,617][00404] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 54.4. Samples: 272. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2025-01-19 23:58:46,640][00404] Avg episode reward: [(0, '2.853')]
[2025-01-19 23:58:46,886][02261] Decorrelating experience for 96 frames...
[2025-01-19 23:58:48,426][02258] Decorrelating experience for 64 frames...
[2025-01-19 23:58:48,668][02260] Decorrelating experience for 96 frames...
[2025-01-19 23:58:49,582][02262] Decorrelating experience for 96 frames...
[2025-01-19 23:58:51,159][02242] Signal inference workers to stop experience collection...
[2025-01-19 23:58:51,170][02255] InferenceWorker_p0-w0: stopping experience collection
[2025-01-19 23:58:51,581][02258] Decorrelating experience for 96 frames...
[2025-01-19 23:58:51,617][00404] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 216.8. Samples: 2168. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2025-01-19 23:58:51,620][00404] Avg episode reward: [(0, '2.993')]
[2025-01-19 23:58:53,516][02242] Signal inference workers to resume experience collection...
[2025-01-19 23:58:53,517][02255] InferenceWorker_p0-w0: resuming experience collection
[2025-01-19 23:58:56,617][00404] Fps is (10 sec: 2048.0, 60 sec: 1365.3, 300 sec: 1365.3). Total num frames: 20480. Throughput: 0: 332.3. Samples: 4984. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
[2025-01-19 23:58:56,621][00404] Avg episode reward: [(0, '3.284')]
[2025-01-19 23:59:00,759][02255] Updated weights for policy 0, policy_version 10 (0.0020)
[2025-01-19 23:59:01,617][00404] Fps is (10 sec: 4096.0, 60 sec: 2048.0, 300 sec: 2048.0). Total num frames: 40960. Throughput: 0: 430.4. Samples: 8608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:01,620][00404] Avg episode reward: [(0, '3.966')]
[2025-01-19 23:59:06,620][00404] Fps is (10 sec: 4095.0, 60 sec: 2457.4, 300 sec: 2457.4). Total num frames: 61440. Throughput: 0: 595.2. Samples: 14882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:06,622][00404] Avg episode reward: [(0, '4.311')]
[2025-01-19 23:59:11,617][00404] Fps is (10 sec: 3276.8, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 73728. Throughput: 0: 634.3. Samples: 19030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:11,623][00404] Avg episode reward: [(0, '4.319')]
[2025-01-19 23:59:12,911][02255] Updated weights for policy 0, policy_version 20 (0.0035)
[2025-01-19 23:59:16,617][00404] Fps is (10 sec: 3277.6, 60 sec: 2691.7, 300 sec: 2691.7). Total num frames: 94208. Throughput: 0: 629.4. Samples: 22028. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:16,621][00404] Avg episode reward: [(0, '4.410')]
[2025-01-19 23:59:21,618][00404] Fps is (10 sec: 4505.6, 60 sec: 2969.6, 300 sec: 2969.6). Total num frames: 118784. Throughput: 0: 728.8. Samples: 29154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:21,620][00404] Avg episode reward: [(0, '4.409')]
[2025-01-19 23:59:21,626][02242] Saving new best policy, reward=4.409!
[2025-01-19 23:59:21,979][02255] Updated weights for policy 0, policy_version 30 (0.0012)
[2025-01-19 23:59:26,617][00404] Fps is (10 sec: 4096.0, 60 sec: 3003.7, 300 sec: 3003.7). Total num frames: 135168. Throughput: 0: 760.9. Samples: 34242. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-19 23:59:26,623][00404] Avg episode reward: [(0, '4.435')]
[2025-01-19 23:59:26,638][02242] Saving new best policy, reward=4.435!
[2025-01-19 23:59:31,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3113.0, 300 sec: 3113.0). Total num frames: 155648. Throughput: 0: 809.8. Samples: 36712. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-19 23:59:31,619][00404] Avg episode reward: [(0, '4.439')]
[2025-01-19 23:59:31,624][02242] Saving new best policy, reward=4.439!
[2025-01-19 23:59:33,010][02255] Updated weights for policy 0, policy_version 40 (0.0019)
[2025-01-19 23:59:36,617][00404] Fps is (10 sec: 4505.6, 60 sec: 3276.8, 300 sec: 3276.8). Total num frames: 180224. Throughput: 0: 928.4. Samples: 43948. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-19 23:59:36,620][00404] Avg episode reward: [(0, '4.396')]
[2025-01-19 23:59:41,617][00404] Fps is (10 sec: 4096.0, 60 sec: 3276.8, 300 sec: 3276.8). Total num frames: 196608. Throughput: 0: 1000.1. Samples: 49988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-19 23:59:41,624][00404] Avg episode reward: [(0, '4.421')]
[2025-01-19 23:59:43,449][02255] Updated weights for policy 0, policy_version 50 (0.0016)
[2025-01-19 23:59:46,618][00404] Fps is (10 sec: 3276.7, 60 sec: 3549.9, 300 sec: 3276.8). Total num frames: 212992. Throughput: 0: 968.3. Samples: 52180. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-19 23:59:46,620][00404] Avg episode reward: [(0, '4.371')]
[2025-01-19 23:59:51,617][00404] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3393.8). Total num frames: 237568. Throughput: 0: 979.6. Samples: 58962. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2025-01-19 23:59:51,625][00404] Avg episode reward: [(0, '4.486')]
[2025-01-19 23:59:51,628][02242] Saving new best policy, reward=4.486!
[2025-01-19 23:59:52,725][02255] Updated weights for policy 0, policy_version 60 (0.0016)
[2025-01-19 23:59:56,618][00404] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3495.2). Total num frames: 262144. Throughput: 0: 1040.7. Samples: 65860. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
[2025-01-19 23:59:56,621][00404] Avg episode reward: [(0, '4.693')]
[2025-01-19 23:59:56,629][02242] Saving new best policy, reward=4.693!
[2025-01-20 00:00:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3430.4). Total num frames: 274432. Throughput: 0: 1023.6. Samples: 68088. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:00:01,626][00404] Avg episode reward: [(0, '4.609')]
[2025-01-20 00:00:04,041][02255] Updated weights for policy 0, policy_version 70 (0.0028)
[2025-01-20 00:00:06,617][00404] Fps is (10 sec: 3686.5, 60 sec: 3959.6, 300 sec: 3517.7). Total num frames: 299008. Throughput: 0: 986.2. Samples: 73532. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2025-01-20 00:00:06,634][00404] Avg episode reward: [(0, '4.467')]
[2025-01-20 00:00:06,649][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000073_299008.pth...
[2025-01-20 00:00:11,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3549.9). Total num frames: 319488. Throughput: 0: 1030.4. Samples: 80612. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:00:11,619][00404] Avg episode reward: [(0, '4.396')]
[2025-01-20 00:00:12,598][02255] Updated weights for policy 0, policy_version 80 (0.0012)
[2025-01-20 00:00:16,619][00404] Fps is (10 sec: 4095.5, 60 sec: 4095.9, 300 sec: 3578.6). Total num frames: 339968. Throughput: 0: 1043.9. Samples: 83690. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:00:16,628][00404] Avg episode reward: [(0, '4.515')]
[2025-01-20 00:00:21,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3563.5). Total num frames: 356352. Throughput: 0: 986.9. Samples: 88358. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:00:21,621][00404] Avg episode reward: [(0, '4.479')]
[2025-01-20 00:00:23,717][02255] Updated weights for policy 0, policy_version 90 (0.0016)
[2025-01-20 00:00:26,618][00404] Fps is (10 sec: 4096.5, 60 sec: 4096.0, 300 sec: 3627.9). Total num frames: 380928. Throughput: 0: 1015.3. Samples: 95678. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:00:26,621][00404] Avg episode reward: [(0, '4.259')]
[2025-01-20 00:00:31,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3649.2). Total num frames: 401408. Throughput: 0: 1047.4. Samples: 99314. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:00:31,624][00404] Avg episode reward: [(0, '4.427')]
[2025-01-20 00:00:33,416][02255] Updated weights for policy 0, policy_version 100 (0.0039)
[2025-01-20 00:00:36,621][00404] Fps is (10 sec: 3685.2, 60 sec: 3959.2, 300 sec: 3632.9). Total num frames: 417792. Throughput: 0: 1002.8. Samples: 104092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:00:36,624][00404] Avg episode reward: [(0, '4.628')]
[2025-01-20 00:00:41,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3686.4). Total num frames: 442368. Throughput: 0: 990.5. Samples: 110430. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:00:41,620][00404] Avg episode reward: [(0, '4.574')]
[2025-01-20 00:00:43,232][02255] Updated weights for policy 0, policy_version 110 (0.0015)
[2025-01-20 00:00:46,617][00404] Fps is (10 sec: 4916.9, 60 sec: 4232.6, 300 sec: 3735.6). Total num frames: 466944. Throughput: 0: 1021.7. Samples: 114064. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:00:46,625][00404] Avg episode reward: [(0, '4.494')]
[2025-01-20 00:00:51,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3686.4). Total num frames: 479232. Throughput: 0: 1033.2. Samples: 120028. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:00:51,623][00404] Avg episode reward: [(0, '4.704')]
[2025-01-20 00:00:51,626][02242] Saving new best policy, reward=4.704!
[2025-01-20 00:00:54,293][02255] Updated weights for policy 0, policy_version 120 (0.0021)
[2025-01-20 00:00:56,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3701.6). Total num frames: 499712. Throughput: 0: 995.7. Samples: 125420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:00:56,620][00404] Avg episode reward: [(0, '4.587')]
[2025-01-20 00:01:01,618][00404] Fps is (10 sec: 4505.5, 60 sec: 4164.3, 300 sec: 3744.9). Total num frames: 524288. Throughput: 0: 1009.4. Samples: 129110. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:01:01,622][00404] Avg episode reward: [(0, '4.479')]
[2025-01-20 00:01:02,722][02255] Updated weights for policy 0, policy_version 130 (0.0025)
[2025-01-20 00:01:06,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3757.0). Total num frames: 544768. Throughput: 0: 1058.1. Samples: 135974. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:01:06,622][00404] Avg episode reward: [(0, '4.443')]
[2025-01-20 00:01:11,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 3741.0). Total num frames: 561152. Throughput: 0: 996.4. Samples: 140516. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:01:11,623][00404] Avg episode reward: [(0, '4.428')]
[2025-01-20 00:01:13,913][02255] Updated weights for policy 0, policy_version 140 (0.0019)
[2025-01-20 00:01:16,618][00404] Fps is (10 sec: 4095.9, 60 sec: 4096.1, 300 sec: 3778.9). Total num frames: 585728. Throughput: 0: 991.2. Samples: 143918. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:01:16,624][00404] Avg episode reward: [(0, '4.541')]
[2025-01-20 00:01:21,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3788.8). Total num frames: 606208. Throughput: 0: 1047.3. Samples: 151218. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:01:21,625][00404] Avg episode reward: [(0, '4.494')]
[2025-01-20 00:01:23,094][02255] Updated weights for policy 0, policy_version 150 (0.0013)
[2025-01-20 00:01:26,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 3773.3). Total num frames: 622592. Throughput: 0: 1019.8. Samples: 156320. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:01:26,622][00404] Avg episode reward: [(0, '4.427')]
[2025-01-20 00:01:31,617][00404] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3734.6). Total num frames: 634880. Throughput: 0: 971.6. Samples: 157784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:01:31,619][00404] Avg episode reward: [(0, '4.470')]
[2025-01-20 00:01:35,819][02255] Updated weights for policy 0, policy_version 160 (0.0030)
[2025-01-20 00:01:36,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3959.7, 300 sec: 3744.9). Total num frames: 655360. Throughput: 0: 962.7. Samples: 163350. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:01:36,619][00404] Avg episode reward: [(0, '4.656')]
[2025-01-20 00:01:41,618][00404] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3754.7). Total num frames: 675840. Throughput: 0: 978.1. Samples: 169436. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:01:41,627][00404] Avg episode reward: [(0, '4.464')]
[2025-01-20 00:01:46,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3741.8). Total num frames: 692224. Throughput: 0: 946.4. Samples: 171700. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:01:46,620][00404] Avg episode reward: [(0, '4.624')]
[2025-01-20 00:01:46,955][02255] Updated weights for policy 0, policy_version 170 (0.0025)
[2025-01-20 00:01:51,617][00404] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3772.6). Total num frames: 716800. Throughput: 0: 942.3. Samples: 178378. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:01:51,624][00404] Avg episode reward: [(0, '4.498')]
[2025-01-20 00:01:55,260][02255] Updated weights for policy 0, policy_version 180 (0.0012)
[2025-01-20 00:01:56,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3801.9). Total num frames: 741376. Throughput: 0: 999.6. Samples: 185500. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:01:56,622][00404] Avg episode reward: [(0, '4.281')]
[2025-01-20 00:02:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3768.3). Total num frames: 753664. Throughput: 0: 974.3. Samples: 187762. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:02:01,623][00404] Avg episode reward: [(0, '4.438')]
[2025-01-20 00:02:06,446][02255] Updated weights for policy 0, policy_version 190 (0.0012)
[2025-01-20 00:02:06,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3796.3). Total num frames: 778240. Throughput: 0: 934.4. Samples: 193264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:02:06,623][00404] Avg episode reward: [(0, '4.771')]
[2025-01-20 00:02:06,633][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000190_778240.pth...
[2025-01-20 00:02:06,764][02242] Saving new best policy, reward=4.771!
[2025-01-20 00:02:11,617][00404] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3803.4). Total num frames: 798720. Throughput: 0: 978.7. Samples: 200362. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:02:11,624][00404] Avg episode reward: [(0, '4.718')]
[2025-01-20 00:02:16,622][00404] Fps is (10 sec: 3684.7, 60 sec: 3822.7, 300 sec: 3791.1). Total num frames: 815104. Throughput: 0: 1010.3. Samples: 203252. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:02:16,624][00404] Avg episode reward: [(0, '4.563')]
[2025-01-20 00:02:16,808][02255] Updated weights for policy 0, policy_version 200 (0.0025)
[2025-01-20 00:02:21,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3798.1). Total num frames: 835584. Throughput: 0: 988.0. Samples: 207808. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:02:21,623][00404] Avg episode reward: [(0, '4.579')]
[2025-01-20 00:02:26,450][02255] Updated weights for policy 0, policy_version 210 (0.0015)
[2025-01-20 00:02:26,617][00404] Fps is (10 sec: 4507.6, 60 sec: 3959.5, 300 sec: 3822.9). Total num frames: 860160. Throughput: 0: 1013.6. Samples: 215050. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:02:26,624][00404] Avg episode reward: [(0, '4.521')]
[2025-01-20 00:02:31,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3828.9). Total num frames: 880640. Throughput: 0: 1042.7. Samples: 218622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:02:31,626][00404] Avg episode reward: [(0, '4.573')]
[2025-01-20 00:02:36,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3799.7). Total num frames: 892928. Throughput: 0: 1002.1. Samples: 223472. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:02:36,626][00404] Avg episode reward: [(0, '4.615')]
[2025-01-20 00:02:38,109][02255] Updated weights for policy 0, policy_version 220 (0.0028)
[2025-01-20 00:02:41,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3805.9). Total num frames: 913408. Throughput: 0: 968.0. Samples: 229060. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:02:41,622][00404] Avg episode reward: [(0, '4.756')]
[2025-01-20 00:02:46,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3828.5). Total num frames: 937984. Throughput: 0: 996.3. Samples: 232596. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:02:46,620][00404] Avg episode reward: [(0, '4.725')]
[2025-01-20 00:02:46,999][02255] Updated weights for policy 0, policy_version 230 (0.0020)
[2025-01-20 00:02:51,618][00404] Fps is (10 sec: 4095.8, 60 sec: 3959.4, 300 sec: 3817.5). Total num frames: 954368. Throughput: 0: 997.7. Samples: 238160. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:02:51,620][00404] Avg episode reward: [(0, '4.630')]
[2025-01-20 00:02:56,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3806.9). Total num frames: 970752. Throughput: 0: 955.2. Samples: 243346. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:02:56,622][00404] Avg episode reward: [(0, '4.734')]
[2025-01-20 00:02:58,526][02255] Updated weights for policy 0, policy_version 240 (0.0027)
[2025-01-20 00:03:01,617][00404] Fps is (10 sec: 4096.2, 60 sec: 4027.7, 300 sec: 3828.2). Total num frames: 995328. Throughput: 0: 969.3. Samples: 246868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:03:01,620][00404] Avg episode reward: [(0, '4.829')]
[2025-01-20 00:03:01,624][02242] Saving new best policy, reward=4.829!
[2025-01-20 00:03:06,617][00404] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3833.2). Total num frames: 1015808. Throughput: 0: 1016.4. Samples: 253546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:03:06,619][00404] Avg episode reward: [(0, '4.892')]
[2025-01-20 00:03:06,633][02242] Saving new best policy, reward=4.892!
[2025-01-20 00:03:08,939][02255] Updated weights for policy 0, policy_version 250 (0.0035)
[2025-01-20 00:03:11,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3807.8). Total num frames: 1028096. Throughput: 0: 952.2. Samples: 257900. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:03:11,622][00404] Avg episode reward: [(0, '4.835')]
[2025-01-20 00:03:16,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.8, 300 sec: 3827.9). Total num frames: 1052672. Throughput: 0: 947.6. Samples: 261262. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:03:16,619][00404] Avg episode reward: [(0, '4.689')]
[2025-01-20 00:03:18,699][02255] Updated weights for policy 0, policy_version 260 (0.0026)
[2025-01-20 00:03:21,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3847.3). Total num frames: 1077248. Throughput: 0: 998.6. Samples: 268408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:03:21,623][00404] Avg episode reward: [(0, '4.738')]
[2025-01-20 00:03:26,620][00404] Fps is (10 sec: 4095.0, 60 sec: 3891.0, 300 sec: 3837.3). Total num frames: 1093632. Throughput: 0: 988.5. Samples: 273544. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:03:26,625][00404] Avg episode reward: [(0, '4.834')]
[2025-01-20 00:03:29,838][02255] Updated weights for policy 0, policy_version 270 (0.0021)
[2025-01-20 00:03:31,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3841.8). Total num frames: 1114112. Throughput: 0: 963.7. Samples: 275964. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:03:31,619][00404] Avg episode reward: [(0, '5.025')]
[2025-01-20 00:03:31,626][02242] Saving new best policy, reward=5.025!
[2025-01-20 00:03:36,617][00404] Fps is (10 sec: 4097.0, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 1134592. Throughput: 0: 990.8. Samples: 282744. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:03:36,623][00404] Avg episode reward: [(0, '5.031')]
[2025-01-20 00:03:36,634][02242] Saving new best policy, reward=5.031!
[2025-01-20 00:03:39,540][02255] Updated weights for policy 0, policy_version 280 (0.0016)
[2025-01-20 00:03:41,623][00404] Fps is (10 sec: 3684.4, 60 sec: 3959.1, 300 sec: 3901.5). Total num frames: 1150976. Throughput: 0: 997.4. Samples: 288234. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:03:41,627][00404] Avg episode reward: [(0, '5.128')]
[2025-01-20 00:03:41,629][02242] Saving new best policy, reward=5.128!
[2025-01-20 00:03:46,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3957.2). Total num frames: 1167360. Throughput: 0: 964.9. Samples: 290288. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:03:46,619][00404] Avg episode reward: [(0, '5.078')]
[2025-01-20 00:03:50,580][02255] Updated weights for policy 0, policy_version 290 (0.0016)
[2025-01-20 00:03:51,617][00404] Fps is (10 sec: 4098.2, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 1191936. Throughput: 0: 958.9. Samples: 296696. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:03:51,626][00404] Avg episode reward: [(0, '5.103')]
[2025-01-20 00:03:56,621][00404] Fps is (10 sec: 4504.0, 60 sec: 4027.5, 300 sec: 3971.0). Total num frames: 1212416. Throughput: 0: 1023.5. Samples: 303960. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:03:56,631][00404] Avg episode reward: [(0, '4.993')]
[2025-01-20 00:04:00,859][02255] Updated weights for policy 0, policy_version 300 (0.0021)
[2025-01-20 00:04:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3957.2). Total num frames: 1228800. Throughput: 0: 1000.8. Samples: 306298. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2025-01-20 00:04:01,625][00404] Avg episode reward: [(0, '4.747')]
[2025-01-20 00:04:06,617][00404] Fps is (10 sec: 4097.4, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1253376. Throughput: 0: 970.0. Samples: 312056. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:04:06,619][00404] Avg episode reward: [(0, '4.985')]
[2025-01-20 00:04:06,632][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000306_1253376.pth...
[2025-01-20 00:04:06,762][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000073_299008.pth
[2025-01-20 00:04:09,796][02255] Updated weights for policy 0, policy_version 310 (0.0021)
[2025-01-20 00:04:11,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4164.3, 300 sec: 4012.7). Total num frames: 1277952. Throughput: 0: 1013.9. Samples: 319168. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:04:11,620][00404] Avg episode reward: [(0, '4.979')]
[2025-01-20 00:04:16,620][00404] Fps is (10 sec: 3685.5, 60 sec: 3959.3, 300 sec: 3971.0). Total num frames: 1290240. Throughput: 0: 1022.6. Samples: 321982. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:04:16,622][00404] Avg episode reward: [(0, '4.890')]
[2025-01-20 00:04:21,247][02255] Updated weights for policy 0, policy_version 320 (0.0023)
[2025-01-20 00:04:21,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3984.9). Total num frames: 1310720. Throughput: 0: 975.1. Samples: 326622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:04:21,620][00404] Avg episode reward: [(0, '4.814')]
[2025-01-20 00:04:26,617][00404] Fps is (10 sec: 4506.7, 60 sec: 4027.9, 300 sec: 3998.8). Total num frames: 1335296. Throughput: 0: 1015.8. Samples: 333940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:04:26,620][00404] Avg episode reward: [(0, '5.044')]
[2025-01-20 00:04:29,730][02255] Updated weights for policy 0, policy_version 330 (0.0019)
[2025-01-20 00:04:31,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 1355776. Throughput: 0: 1051.4. Samples: 337600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:04:31,622][00404] Avg episode reward: [(0, '4.943')]
[2025-01-20 00:04:36,618][00404] Fps is (10 sec: 3686.3, 60 sec: 3959.4, 300 sec: 3984.9). Total num frames: 1372160. Throughput: 0: 1013.8. Samples: 342316. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:04:36,625][00404] Avg episode reward: [(0, '4.721')]
[2025-01-20 00:04:40,622][02255] Updated weights for policy 0, policy_version 340 (0.0014)
[2025-01-20 00:04:41,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4028.1, 300 sec: 3998.8). Total num frames: 1392640. Throughput: 0: 1002.4. Samples: 349066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:04:41,620][00404] Avg episode reward: [(0, '4.716')]
[2025-01-20 00:04:46,620][00404] Fps is (10 sec: 4504.5, 60 sec: 4164.1, 300 sec: 3998.8). Total num frames: 1417216. Throughput: 0: 1031.6. Samples: 352724. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:04:46,623][00404] Avg episode reward: [(0, '4.635')]
[2025-01-20 00:04:50,845][02255] Updated weights for policy 0, policy_version 350 (0.0020)
[2025-01-20 00:04:51,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3971.0). Total num frames: 1433600. Throughput: 0: 1023.8. Samples: 358128. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2025-01-20 00:04:51,621][00404] Avg episode reward: [(0, '4.867')]
[2025-01-20 00:04:56,617][00404] Fps is (10 sec: 3687.4, 60 sec: 4028.0, 300 sec: 3998.8). Total num frames: 1454080. Throughput: 0: 998.0. Samples: 364076. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:04:56,626][00404] Avg episode reward: [(0, '4.983')]
[2025-01-20 00:05:00,164][02255] Updated weights for policy 0, policy_version 360 (0.0013)
[2025-01-20 00:05:01,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3998.8). Total num frames: 1478656. Throughput: 0: 1017.3. Samples: 367758. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:01,623][00404] Avg episode reward: [(0, '4.885')]
[2025-01-20 00:05:06,618][00404] Fps is (10 sec: 4505.4, 60 sec: 4096.0, 300 sec: 3998.8). Total num frames: 1499136. Throughput: 0: 1054.7. Samples: 374086. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:05:06,620][00404] Avg episode reward: [(0, '4.973')]
[2025-01-20 00:05:11,027][02255] Updated weights for policy 0, policy_version 370 (0.0025)
[2025-01-20 00:05:11,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 1515520. Throughput: 0: 1009.7. Samples: 379376. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:05:11,621][00404] Avg episode reward: [(0, '4.897')]
[2025-01-20 00:05:16,617][00404] Fps is (10 sec: 4096.2, 60 sec: 4164.4, 300 sec: 4012.7). Total num frames: 1540096. Throughput: 0: 1007.0. Samples: 382914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:05:16,620][00404] Avg episode reward: [(0, '4.572')]
[2025-01-20 00:05:19,471][02255] Updated weights for policy 0, policy_version 380 (0.0012)
[2025-01-20 00:05:21,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3998.8). Total num frames: 1560576. Throughput: 0: 1061.5. Samples: 390084. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:21,622][00404] Avg episode reward: [(0, '4.630')]
[2025-01-20 00:05:26,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 1576960. Throughput: 0: 1011.6. Samples: 394590. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:05:26,624][00404] Avg episode reward: [(0, '4.779')]
[2025-01-20 00:05:30,436][02255] Updated weights for policy 0, policy_version 390 (0.0014)
[2025-01-20 00:05:31,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4012.7). Total num frames: 1601536. Throughput: 0: 1007.1. Samples: 398040. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:05:31,624][00404] Avg episode reward: [(0, '4.792')]
[2025-01-20 00:05:36,620][00404] Fps is (10 sec: 4914.0, 60 sec: 4232.4, 300 sec: 4012.7). Total num frames: 1626112. Throughput: 0: 1050.3. Samples: 405392. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:36,623][00404] Avg episode reward: [(0, '4.760')]
[2025-01-20 00:05:40,417][02255] Updated weights for policy 0, policy_version 400 (0.0016)
[2025-01-20 00:05:41,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 3971.0). Total num frames: 1638400. Throughput: 0: 1033.6. Samples: 410588. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:41,620][00404] Avg episode reward: [(0, '4.684')]
[2025-01-20 00:05:46,617][00404] Fps is (10 sec: 3687.3, 60 sec: 4096.2, 300 sec: 4012.7). Total num frames: 1662976. Throughput: 0: 1007.7. Samples: 413106. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:05:46,622][00404] Avg episode reward: [(0, '4.742')]
[2025-01-20 00:05:49,923][02255] Updated weights for policy 0, policy_version 410 (0.0021)
[2025-01-20 00:05:51,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4012.7). Total num frames: 1683456. Throughput: 0: 1030.1. Samples: 420438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:51,626][00404] Avg episode reward: [(0, '5.028')]
[2025-01-20 00:05:56,618][00404] Fps is (10 sec: 4095.9, 60 sec: 4164.2, 300 sec: 3998.8). Total num frames: 1703936. Throughput: 0: 1050.9. Samples: 426666. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:05:56,620][00404] Avg episode reward: [(0, '5.183')]
[2025-01-20 00:05:56,628][02242] Saving new best policy, reward=5.183!
[2025-01-20 00:06:00,763][02255] Updated weights for policy 0, policy_version 420 (0.0011)
[2025-01-20 00:06:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 1720320. Throughput: 0: 1023.7. Samples: 428982. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:06:01,625][00404] Avg episode reward: [(0, '5.152')]
[2025-01-20 00:06:06,618][00404] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4012.7). Total num frames: 1744896. Throughput: 0: 1014.2. Samples: 435722. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:06,624][00404] Avg episode reward: [(0, '5.682')]
[2025-01-20 00:06:06,650][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000427_1748992.pth...
[2025-01-20 00:06:06,761][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000190_778240.pth
[2025-01-20 00:06:06,781][02242] Saving new best policy, reward=5.682!
[2025-01-20 00:06:09,357][02255] Updated weights for policy 0, policy_version 430 (0.0024)
[2025-01-20 00:06:11,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4012.7). Total num frames: 1769472. Throughput: 0: 1067.8. Samples: 442640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:06:11,620][00404] Avg episode reward: [(0, '6.110')]
[2025-01-20 00:06:11,623][02242] Saving new best policy, reward=6.110!
[2025-01-20 00:06:16,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 1781760. Throughput: 0: 1038.0. Samples: 444750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:16,627][00404] Avg episode reward: [(0, '6.091')]
[2025-01-20 00:06:20,665][02255] Updated weights for policy 0, policy_version 440 (0.0026)
[2025-01-20 00:06:21,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4012.7). Total num frames: 1806336. Throughput: 0: 1001.0. Samples: 450434. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:21,623][00404] Avg episode reward: [(0, '5.340')]
[2025-01-20 00:06:26,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4054.3). Total num frames: 1830912. Throughput: 0: 1046.5. Samples: 457682. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:26,619][00404] Avg episode reward: [(0, '5.255')]
[2025-01-20 00:06:30,325][02255] Updated weights for policy 0, policy_version 450 (0.0024)
[2025-01-20 00:06:31,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4040.5). Total num frames: 1847296. Throughput: 0: 1053.0. Samples: 460492. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:31,620][00404] Avg episode reward: [(0, '5.264')]
[2025-01-20 00:06:36,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3959.6, 300 sec: 4026.6). Total num frames: 1863680. Throughput: 0: 995.0. Samples: 465212. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:06:36,620][00404] Avg episode reward: [(0, '5.407')]
[2025-01-20 00:06:40,502][02255] Updated weights for policy 0, policy_version 460 (0.0017)
[2025-01-20 00:06:41,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4054.3). Total num frames: 1888256. Throughput: 0: 1013.1. Samples: 472254. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2025-01-20 00:06:41,623][00404] Avg episode reward: [(0, '5.282')]
[2025-01-20 00:06:46,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 4026.6). Total num frames: 1904640. Throughput: 0: 1038.5. Samples: 475714. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:06:46,619][00404] Avg episode reward: [(0, '5.349')]
[2025-01-20 00:06:51,618][00404] Fps is (10 sec: 3276.7, 60 sec: 3959.5, 300 sec: 3998.8). Total num frames: 1921024. Throughput: 0: 990.6. Samples: 480298. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:06:51,619][00404] Avg episode reward: [(0, '5.145')]
[2025-01-20 00:06:51,646][02255] Updated weights for policy 0, policy_version 470 (0.0016)
[2025-01-20 00:06:56,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 4040.5). Total num frames: 1945600. Throughput: 0: 986.4. Samples: 487030. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:06:56,620][00404] Avg episode reward: [(0, '5.437')]
[2025-01-20 00:07:00,264][02255] Updated weights for policy 0, policy_version 480 (0.0020)
[2025-01-20 00:07:01,617][00404] Fps is (10 sec: 4915.3, 60 sec: 4164.3, 300 sec: 4040.5). Total num frames: 1970176. Throughput: 0: 1019.4. Samples: 490622. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:07:01,620][00404] Avg episode reward: [(0, '5.835')]
[2025-01-20 00:07:06,618][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 1982464. Throughput: 0: 1008.4. Samples: 495810. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2025-01-20 00:07:06,620][00404] Avg episode reward: [(0, '6.042')]
[2025-01-20 00:07:11,617][00404] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 4026.6). Total num frames: 2002944. Throughput: 0: 976.8. Samples: 501638. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2025-01-20 00:07:11,619][00404] Avg episode reward: [(0, '6.544')]
[2025-01-20 00:07:11,626][02242] Saving new best policy, reward=6.544!
[2025-01-20 00:07:11,640][02255] Updated weights for policy 0, policy_version 490 (0.0018)
[2025-01-20 00:07:16,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 4040.5). Total num frames: 2027520. Throughput: 0: 990.3. Samples: 505056. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2025-01-20 00:07:16,619][00404] Avg episode reward: [(0, '6.561')]
[2025-01-20 00:07:16,629][02242] Saving new best policy, reward=6.561!
[2025-01-20 00:07:21,617][00404] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 4012.7). Total num frames: 2043904. Throughput: 0: 1021.8. Samples: 511194. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:07:21,623][00404] Avg episode reward: [(0, '6.385')]
[2025-01-20 00:07:21,829][02255] Updated weights for policy 0, policy_version 500 (0.0013)
[2025-01-20 00:07:26,618][00404] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 4012.7). Total num frames: 2064384. Throughput: 0: 983.4. Samples: 516508. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:07:26,624][00404] Avg episode reward: [(0, '6.022')]
[2025-01-20 00:07:31,069][02255] Updated weights for policy 0, policy_version 510 (0.0023)
[2025-01-20 00:07:31,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 4054.3). Total num frames: 2088960. Throughput: 0: 986.6. Samples: 520112. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:07:31,624][00404] Avg episode reward: [(0, '6.918')]
[2025-01-20 00:07:31,630][02242] Saving new best policy, reward=6.918!
[2025-01-20 00:07:36,617][00404] Fps is (10 sec: 4505.7, 60 sec: 4096.0, 300 sec: 4054.3). Total num frames: 2109440. Throughput: 0: 1039.3. Samples: 527066. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:07:36,625][00404] Avg episode reward: [(0, '7.053')]
[2025-01-20 00:07:36,652][02242] Saving new best policy, reward=7.053!
[2025-01-20 00:07:41,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4026.6). Total num frames: 2125824. Throughput: 0: 990.0. Samples: 531580. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:07:41,626][00404] Avg episode reward: [(0, '6.695')]
[2025-01-20 00:07:42,181][02255] Updated weights for policy 0, policy_version 520 (0.0023)
[2025-01-20 00:07:46,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4054.4). Total num frames: 2150400. Throughput: 0: 983.2. Samples: 534866. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:07:46,624][00404] Avg episode reward: [(0, '6.598')]
[2025-01-20 00:07:50,899][02255] Updated weights for policy 0, policy_version 530 (0.0021)
[2025-01-20 00:07:51,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 4068.2). Total num frames: 2170880. Throughput: 0: 1028.5. Samples: 542094. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:07:51,624][00404] Avg episode reward: [(0, '6.928')]
[2025-01-20 00:07:56,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4040.5). Total num frames: 2187264. Throughput: 0: 1015.1. Samples: 547316. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:07:56,620][00404] Avg episode reward: [(0, '6.958')]
[2025-01-20 00:08:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 4040.5). Total num frames: 2207744. Throughput: 0: 1000.4. Samples: 550076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:08:01,624][00404] Avg episode reward: [(0, '7.051')]
[2025-01-20 00:08:01,791][02255] Updated weights for policy 0, policy_version 540 (0.0013)
[2025-01-20 00:08:06,618][00404] Fps is (10 sec: 4505.5, 60 sec: 4164.3, 300 sec: 4082.1). Total num frames: 2232320. Throughput: 0: 1028.8. Samples: 557492. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:08:06,624][00404] Avg episode reward: [(0, '7.038')]
[2025-01-20 00:08:06,693][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000546_2236416.pth...
[2025-01-20 00:08:06,808][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000306_1253376.pth
[2025-01-20 00:08:10,980][02255] Updated weights for policy 0, policy_version 550 (0.0031)
[2025-01-20 00:08:11,623][00404] Fps is (10 sec: 4503.2, 60 sec: 4163.9, 300 sec: 4068.2). Total num frames: 2252800. Throughput: 0: 1044.6. Samples: 563520. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:08:11,634][00404] Avg episode reward: [(0, '7.513')]
[2025-01-20 00:08:11,645][02242] Saving new best policy, reward=7.513!
[2025-01-20 00:08:16,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4027.7, 300 sec: 4040.5). Total num frames: 2269184. Throughput: 0: 1012.7. Samples: 565684. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:08:16,626][00404] Avg episode reward: [(0, '7.969')]
[2025-01-20 00:08:16,635][02242] Saving new best policy, reward=7.969!
[2025-01-20 00:08:21,149][02255] Updated weights for policy 0, policy_version 560 (0.0027)
[2025-01-20 00:08:21,617][00404] Fps is (10 sec: 4098.2, 60 sec: 4164.3, 300 sec: 4068.3). Total num frames: 2293760. Throughput: 0: 1009.5. Samples: 572492. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:08:21,624][00404] Avg episode reward: [(0, '7.792')]
[2025-01-20 00:08:26,619][00404] Fps is (10 sec: 4504.9, 60 sec: 4164.2, 300 sec: 4068.2). Total num frames: 2314240. Throughput: 0: 1066.9. Samples: 579590. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:08:26,624][00404] Avg episode reward: [(0, '8.176')]
[2025-01-20 00:08:26,703][02242] Saving new best policy, reward=8.176!
[2025-01-20 00:08:31,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4054.3). Total num frames: 2330624. Throughput: 0: 1043.9. Samples: 581842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:08:31,623][00404] Avg episode reward: [(0, '8.343')]
[2025-01-20 00:08:31,626][02242] Saving new best policy, reward=8.343!
[2025-01-20 00:08:32,177][02255] Updated weights for policy 0, policy_version 570 (0.0018)
[2025-01-20 00:08:36,617][00404] Fps is (10 sec: 4096.7, 60 sec: 4096.0, 300 sec: 4082.2). Total num frames: 2355200. Throughput: 0: 1016.1. Samples: 587820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:08:36,625][00404] Avg episode reward: [(0, '8.985')]
[2025-01-20 00:08:36,634][02242] Saving new best policy, reward=8.985!
[2025-01-20 00:08:40,234][02255] Updated weights for policy 0, policy_version 580 (0.0026)
[2025-01-20 00:08:41,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 4109.9). Total num frames: 2379776. Throughput: 0: 1066.3. Samples: 595300. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:08:41,626][00404] Avg episode reward: [(0, '8.462')]
[2025-01-20 00:08:46,622][00404] Fps is (10 sec: 4094.2, 60 sec: 4095.7, 300 sec: 4082.1). Total num frames: 2396160. Throughput: 0: 1063.5. Samples: 597938. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:08:46,625][00404] Avg episode reward: [(0, '8.546')]
[2025-01-20 00:08:51,162][02255] Updated weights for policy 0, policy_version 590 (0.0025)
[2025-01-20 00:08:51,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4082.2). Total num frames: 2416640. Throughput: 0: 1011.5. Samples: 603010. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:08:51,624][00404] Avg episode reward: [(0, '8.636')]
[2025-01-20 00:08:56,617][00404] Fps is (10 sec: 4507.6, 60 sec: 4232.5, 300 sec: 4109.9). Total num frames: 2441216. Throughput: 0: 1044.5. Samples: 610516. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:08:56,624][00404] Avg episode reward: [(0, '9.469')]
[2025-01-20 00:08:56,630][02242] Saving new best policy, reward=9.469!
[2025-01-20 00:08:59,858][02255] Updated weights for policy 0, policy_version 600 (0.0011)
[2025-01-20 00:09:01,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4096.0). Total num frames: 2461696. Throughput: 0: 1077.6. Samples: 614176. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:09:01,621][00404] Avg episode reward: [(0, '8.759')]
[2025-01-20 00:09:06,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4068.2). Total num frames: 2478080. Throughput: 0: 1028.5. Samples: 618774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:09:06,626][00404] Avg episode reward: [(0, '8.432')]
[2025-01-20 00:09:10,394][02255] Updated weights for policy 0, policy_version 610 (0.0016)
[2025-01-20 00:09:11,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.6, 300 sec: 4109.9). Total num frames: 2502656. Throughput: 0: 1028.7. Samples: 625880. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:09:11,619][00404] Avg episode reward: [(0, '8.840')]
[2025-01-20 00:09:16,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4123.8). Total num frames: 2527232. Throughput: 0: 1058.4. Samples: 629470. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2025-01-20 00:09:16,621][00404] Avg episode reward: [(0, '9.758')]
[2025-01-20 00:09:16,629][02242] Saving new best policy, reward=9.758!
[2025-01-20 00:09:20,319][02255] Updated weights for policy 0, policy_version 620 (0.0012)
[2025-01-20 00:09:21,620][00404] Fps is (10 sec: 3685.5, 60 sec: 4095.8, 300 sec: 4082.1). Total num frames: 2539520. Throughput: 0: 1046.0. Samples: 634892. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0)
[2025-01-20 00:09:21,622][00404] Avg episode reward: [(0, '10.490')]
[2025-01-20 00:09:21,652][02242] Saving new best policy, reward=10.490!
[2025-01-20 00:09:26,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4164.4, 300 sec: 4096.0). Total num frames: 2564096. Throughput: 0: 1016.3. Samples: 641032. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:09:26,623][00404] Avg episode reward: [(0, '10.679')]
[2025-01-20 00:09:26,629][02242] Saving new best policy, reward=10.679!
[2025-01-20 00:09:29,663][02255] Updated weights for policy 0, policy_version 630 (0.0017)
[2025-01-20 00:09:31,617][00404] Fps is (10 sec: 4916.4, 60 sec: 4300.8, 300 sec: 4123.8). Total num frames: 2588672. Throughput: 0: 1038.0. Samples: 644642. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:09:31,623][00404] Avg episode reward: [(0, '11.349')]
[2025-01-20 00:09:31,628][02242] Saving new best policy, reward=11.349!
[2025-01-20 00:09:36,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4109.9). Total num frames: 2605056. Throughput: 0: 1066.5. Samples: 651004. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:09:36,623][00404] Avg episode reward: [(0, '10.991')]
[2025-01-20 00:09:40,572][02255] Updated weights for policy 0, policy_version 640 (0.0014)
[2025-01-20 00:09:41,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2625536. Throughput: 0: 1018.4. Samples: 656342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:09:41,620][00404] Avg episode reward: [(0, '11.875')]
[2025-01-20 00:09:41,624][02242] Saving new best policy, reward=11.875!
[2025-01-20 00:09:46,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.9, 300 sec: 4123.8). Total num frames: 2650112. Throughput: 0: 1016.9. Samples: 659938. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:09:46,624][00404] Avg episode reward: [(0, '11.856')]
[2025-01-20 00:09:48,987][02255] Updated weights for policy 0, policy_version 650 (0.0017)
[2025-01-20 00:09:51,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4123.8). Total num frames: 2670592. Throughput: 0: 1075.6. Samples: 667176. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:09:51,624][00404] Avg episode reward: [(0, '12.563')]
[2025-01-20 00:09:51,629][02242] Saving new best policy, reward=12.563!
[2025-01-20 00:09:56,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2686976. Throughput: 0: 1019.3. Samples: 671750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:09:56,620][00404] Avg episode reward: [(0, '13.844')]
[2025-01-20 00:09:56,630][02242] Saving new best policy, reward=13.844!
[2025-01-20 00:09:59,880][02255] Updated weights for policy 0, policy_version 660 (0.0020)
[2025-01-20 00:10:01,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4109.9). Total num frames: 2711552. Throughput: 0: 1017.9. Samples: 675274. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:10:01,620][00404] Avg episode reward: [(0, '12.873')]
[2025-01-20 00:10:06,620][00404] Fps is (10 sec: 4913.9, 60 sec: 4300.6, 300 sec: 4137.6). Total num frames: 2736128. Throughput: 0: 1063.6. Samples: 682754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:10:06,622][00404] Avg episode reward: [(0, '11.676')]
[2025-01-20 00:10:06,635][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000668_2736128.pth...
[2025-01-20 00:10:06,792][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000427_1748992.pth
[2025-01-20 00:10:09,064][02255] Updated weights for policy 0, policy_version 670 (0.0020)
[2025-01-20 00:10:11,618][00404] Fps is (10 sec: 3686.2, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2748416. Throughput: 0: 1044.4. Samples: 688032. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:10:11,626][00404] Avg episode reward: [(0, '11.343')]
[2025-01-20 00:10:16,617][00404] Fps is (10 sec: 3687.4, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 2772992. Throughput: 0: 1025.1. Samples: 690772. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:10:16,620][00404] Avg episode reward: [(0, '11.595')]
[2025-01-20 00:10:18,949][02255] Updated weights for policy 0, policy_version 680 (0.0013)
[2025-01-20 00:10:21,617][00404] Fps is (10 sec: 4915.5, 60 sec: 4301.0, 300 sec: 4137.7). Total num frames: 2797568. Throughput: 0: 1046.8. Samples: 698108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:10:21,623][00404] Avg episode reward: [(0, '12.467')]
[2025-01-20 00:10:26,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4109.9). Total num frames: 2813952. Throughput: 0: 1063.5. Samples: 704198. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:10:26,623][00404] Avg episode reward: [(0, '13.373')]
[2025-01-20 00:10:29,770][02255] Updated weights for policy 0, policy_version 690 (0.0015)
[2025-01-20 00:10:31,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 2834432. Throughput: 0: 1034.3. Samples: 706482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:10:31,623][00404] Avg episode reward: [(0, '13.618')]
[2025-01-20 00:10:36,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 2859008. Throughput: 0: 1030.2. Samples: 713536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:10:36,625][00404] Avg episode reward: [(0, '14.138')]
[2025-01-20 00:10:36,632][02242] Saving new best policy, reward=14.138!
[2025-01-20 00:10:37,947][02255] Updated weights for policy 0, policy_version 700 (0.0015)
[2025-01-20 00:10:41,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4123.8). Total num frames: 2879488. Throughput: 0: 1083.3. Samples: 720500. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:10:41,623][00404] Avg episode reward: [(0, '15.339')]
[2025-01-20 00:10:41,625][02242] Saving new best policy, reward=15.339!
[2025-01-20 00:10:46,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 2895872. Throughput: 0: 1055.6. Samples: 722778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:10:46,621][00404] Avg episode reward: [(0, '14.371')]
[2025-01-20 00:10:49,006][02255] Updated weights for policy 0, policy_version 710 (0.0026)
[2025-01-20 00:10:51,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2920448. Throughput: 0: 1022.5. Samples: 728764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:10:51,620][00404] Avg episode reward: [(0, '12.743')]
[2025-01-20 00:10:56,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4151.5). Total num frames: 2945024. Throughput: 0: 1071.4. Samples: 736244. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:10:56,621][00404] Avg episode reward: [(0, '12.829')]
[2025-01-20 00:10:57,506][02255] Updated weights for policy 0, policy_version 720 (0.0024)
[2025-01-20 00:11:01,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 2961408. Throughput: 0: 1070.0. Samples: 738924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:11:01,619][00404] Avg episode reward: [(0, '13.302')]
[2025-01-20 00:11:06,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.2, 300 sec: 4109.9). Total num frames: 2981888. Throughput: 0: 1020.9. Samples: 744050. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:11:06,623][00404] Avg episode reward: [(0, '14.076')]
[2025-01-20 00:11:08,144][02255] Updated weights for policy 0, policy_version 730 (0.0035)
[2025-01-20 00:11:11,618][00404] Fps is (10 sec: 4505.5, 60 sec: 4300.8, 300 sec: 4151.5). Total num frames: 3006464. Throughput: 0: 1053.9. Samples: 751622. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:11,625][00404] Avg episode reward: [(0, '14.217')]
[2025-01-20 00:11:16,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4123.8). Total num frames: 3022848. Throughput: 0: 1085.7. Samples: 755340. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:16,623][00404] Avg episode reward: [(0, '14.972')]
[2025-01-20 00:11:17,985][02255] Updated weights for policy 0, policy_version 740 (0.0018)
[2025-01-20 00:11:21,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4096.0, 300 sec: 4109.9). Total num frames: 3043328. Throughput: 0: 1030.9. Samples: 759928. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:21,623][00404] Avg episode reward: [(0, '15.097')]
[2025-01-20 00:11:26,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4137.7). Total num frames: 3067904. Throughput: 0: 1035.3. Samples: 767090. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:26,620][00404] Avg episode reward: [(0, '16.263')]
[2025-01-20 00:11:26,631][02242] Saving new best policy, reward=16.263!
[2025-01-20 00:11:27,255][02255] Updated weights for policy 0, policy_version 750 (0.0024)
[2025-01-20 00:11:31,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4165.4). Total num frames: 3092480. Throughput: 0: 1065.7. Samples: 770736. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:11:31,621][00404] Avg episode reward: [(0, '15.570')]
[2025-01-20 00:11:36,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4123.8). Total num frames: 3104768. Throughput: 0: 1057.2. Samples: 776336. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:11:36,624][00404] Avg episode reward: [(0, '15.547')]
[2025-01-20 00:11:37,996][02255] Updated weights for policy 0, policy_version 760 (0.0021)
[2025-01-20 00:11:41,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4164.3, 300 sec: 4151.5). Total num frames: 3129344. Throughput: 0: 1028.5. Samples: 782526. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:11:41,620][00404] Avg episode reward: [(0, '14.403')]
[2025-01-20 00:11:46,327][02255] Updated weights for policy 0, policy_version 770 (0.0014)
[2025-01-20 00:11:46,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4179.3). Total num frames: 3153920. Throughput: 0: 1051.5. Samples: 786242. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:46,620][00404] Avg episode reward: [(0, '13.642')]
[2025-01-20 00:11:51,618][00404] Fps is (10 sec: 4095.6, 60 sec: 4164.2, 300 sec: 4151.5). Total num frames: 3170304. Throughput: 0: 1078.3. Samples: 792576. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:11:51,624][00404] Avg episode reward: [(0, '13.666')]
[2025-01-20 00:11:56,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4137.7). Total num frames: 3190784. Throughput: 0: 1030.1. Samples: 797976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:11:56,620][00404] Avg episode reward: [(0, '12.935')]
[2025-01-20 00:11:57,105][02255] Updated weights for policy 0, policy_version 780 (0.0030)
[2025-01-20 00:12:01,617][00404] Fps is (10 sec: 4506.0, 60 sec: 4232.5, 300 sec: 4179.3). Total num frames: 3215360. Throughput: 0: 1031.3. Samples: 801750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:12:01,624][00404] Avg episode reward: [(0, '14.114')]
[2025-01-20 00:12:05,469][02255] Updated weights for policy 0, policy_version 790 (0.0015)
[2025-01-20 00:12:06,620][00404] Fps is (10 sec: 4504.5, 60 sec: 4232.4, 300 sec: 4179.3). Total num frames: 3235840. Throughput: 0: 1092.1. Samples: 809076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:12:06,622][00404] Avg episode reward: [(0, '15.218')]
[2025-01-20 00:12:06,633][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000790_3235840.pth...
[2025-01-20 00:12:06,785][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000546_2236416.pth
[2025-01-20 00:12:11,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4151.5). Total num frames: 3252224. Throughput: 0: 1032.9. Samples: 813572. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:12:11,620][00404] Avg episode reward: [(0, '15.372')]
[2025-01-20 00:12:16,217][02255] Updated weights for policy 0, policy_version 800 (0.0028)
[2025-01-20 00:12:16,617][00404] Fps is (10 sec: 4097.0, 60 sec: 4232.5, 300 sec: 4179.3). Total num frames: 3276800. Throughput: 0: 1030.6. Samples: 817114. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:12:16,619][00404] Avg episode reward: [(0, '17.245')]
[2025-01-20 00:12:16,627][02242] Saving new best policy, reward=17.245!
[2025-01-20 00:12:21,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4193.2). Total num frames: 3301376. Throughput: 0: 1069.4. Samples: 824458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:12:21,622][00404] Avg episode reward: [(0, '18.139')]
[2025-01-20 00:12:21,625][02242] Saving new best policy, reward=18.139!
[2025-01-20 00:12:26,422][02255] Updated weights for policy 0, policy_version 810 (0.0017)
[2025-01-20 00:12:26,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4165.4). Total num frames: 3317760. Throughput: 0: 1047.2. Samples: 829652. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:12:26,620][00404] Avg episode reward: [(0, '18.175')]
[2025-01-20 00:12:26,629][02242] Saving new best policy, reward=18.175!
[2025-01-20 00:12:31,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4165.4). Total num frames: 3338240. Throughput: 0: 1024.7. Samples: 832354. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:12:31,624][00404] Avg episode reward: [(0, '18.722')]
[2025-01-20 00:12:31,630][02242] Saving new best policy, reward=18.722!
[2025-01-20 00:12:35,286][02255] Updated weights for policy 0, policy_version 820 (0.0014)
[2025-01-20 00:12:36,619][00404] Fps is (10 sec: 4504.9, 60 sec: 4300.7, 300 sec: 4193.2). Total num frames: 3362816. Throughput: 0: 1049.1. Samples: 839788. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:12:36,626][00404] Avg episode reward: [(0, '17.354')]
[2025-01-20 00:12:41,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4179.3). Total num frames: 3383296. Throughput: 0: 1064.7. Samples: 845888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:12:41,620][00404] Avg episode reward: [(0, '15.934')]
[2025-01-20 00:12:46,078][02255] Updated weights for policy 0, policy_version 830 (0.0020)
[2025-01-20 00:12:46,617][00404] Fps is (10 sec: 3686.9, 60 sec: 4096.0, 300 sec: 4165.4). Total num frames: 3399680. Throughput: 0: 1033.6. Samples: 848260. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:12:46,619][00404] Avg episode reward: [(0, '15.900')]
[2025-01-20 00:12:51,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4232.6, 300 sec: 4193.2). Total num frames: 3424256. Throughput: 0: 1025.7. Samples: 855230. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2025-01-20 00:12:51,621][00404] Avg episode reward: [(0, '15.170')]
[2025-01-20 00:12:54,312][02255] Updated weights for policy 0, policy_version 840 (0.0016)
[2025-01-20 00:12:56,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4193.2). Total num frames: 3444736. Throughput: 0: 1078.4. Samples: 862102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:12:56,628][00404] Avg episode reward: [(0, '15.746')]
[2025-01-20 00:13:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4096.0, 300 sec: 4165.4). Total num frames: 3461120. Throughput: 0: 1051.2. Samples: 864418. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:13:01,625][00404] Avg episode reward: [(0, '15.885')]
[2025-01-20 00:13:05,034][02255] Updated weights for policy 0, policy_version 850 (0.0025)
[2025-01-20 00:13:06,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.4, 300 sec: 4179.4). Total num frames: 3485696. Throughput: 0: 1031.0. Samples: 870852. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:06,623][00404] Avg episode reward: [(0, '16.287')]
[2025-01-20 00:13:11,621][00404] Fps is (10 sec: 4913.5, 60 sec: 4300.5, 300 sec: 4207.0). Total num frames: 3510272. Throughput: 0: 1080.5. Samples: 878278. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:11,623][00404] Avg episode reward: [(0, '17.027')]
[2025-01-20 00:13:14,393][02255] Updated weights for policy 0, policy_version 860 (0.0017)
[2025-01-20 00:13:16,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4164.3, 300 sec: 4179.3). Total num frames: 3526656. Throughput: 0: 1077.2. Samples: 880828. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:16,620][00404] Avg episode reward: [(0, '17.269')]
[2025-01-20 00:13:21,617][00404] Fps is (10 sec: 4097.4, 60 sec: 4164.3, 300 sec: 4193.2). Total num frames: 3551232. Throughput: 0: 1033.4. Samples: 886290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:13:21,627][00404] Avg episode reward: [(0, '16.990')]
[2025-01-20 00:13:24,055][02255] Updated weights for policy 0, policy_version 870 (0.0018)
[2025-01-20 00:13:26,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4221.0). Total num frames: 3575808. Throughput: 0: 1064.8. Samples: 893806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:26,625][00404] Avg episode reward: [(0, '17.702')]
[2025-01-20 00:13:31,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 4193.2). Total num frames: 3592192. Throughput: 0: 1086.8. Samples: 897168. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:31,620][00404] Avg episode reward: [(0, '17.188')]
[2025-01-20 00:13:34,615][02255] Updated weights for policy 0, policy_version 880 (0.0024)
[2025-01-20 00:13:36,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4164.4, 300 sec: 4179.3). Total num frames: 3612672. Throughput: 0: 1035.1. Samples: 901810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2025-01-20 00:13:36,629][00404] Avg episode reward: [(0, '16.498')]
[2025-01-20 00:13:41,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4207.1). Total num frames: 3637248. Throughput: 0: 1047.6. Samples: 909244. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:13:41,628][00404] Avg episode reward: [(0, '17.110')]
[2025-01-20 00:13:43,073][02255] Updated weights for policy 0, policy_version 890 (0.0018)
[2025-01-20 00:13:46,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4300.8, 300 sec: 4207.1). Total num frames: 3657728. Throughput: 0: 1080.0. Samples: 913020. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:46,625][00404] Avg episode reward: [(0, '16.387')]
[2025-01-20 00:13:51,618][00404] Fps is (10 sec: 3686.2, 60 sec: 4164.2, 300 sec: 4179.3). Total num frames: 3674112. Throughput: 0: 1051.1. Samples: 918152. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:13:51,620][00404] Avg episode reward: [(0, '16.092')]
[2025-01-20 00:13:53,911][02255] Updated weights for policy 0, policy_version 900 (0.0019)
[2025-01-20 00:13:56,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 4193.2). Total num frames: 3698688. Throughput: 0: 1033.6. Samples: 924788. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:13:56,619][00404] Avg episode reward: [(0, '17.712')]
[2025-01-20 00:14:01,617][00404] Fps is (10 sec: 4915.4, 60 sec: 4369.1, 300 sec: 4221.0). Total num frames: 3723264. Throughput: 0: 1061.0. Samples: 928574. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:01,619][00404] Avg episode reward: [(0, '18.195')]
[2025-01-20 00:14:01,896][02255] Updated weights for policy 0, policy_version 910 (0.0015)
[2025-01-20 00:14:06,617][00404] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 4193.2). Total num frames: 3739648. Throughput: 0: 1077.2. Samples: 934762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:06,628][00404] Avg episode reward: [(0, '18.622')]
[2025-01-20 00:14:06,641][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000913_3739648.pth...
[2025-01-20 00:14:06,806][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000668_2736128.pth
[2025-01-20 00:14:11,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4164.5, 300 sec: 4179.3). Total num frames: 3760128. Throughput: 0: 1038.3. Samples: 940530. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2025-01-20 00:14:11,619][00404] Avg episode reward: [(0, '18.908')]
[2025-01-20 00:14:11,627][02242] Saving new best policy, reward=18.908!
[2025-01-20 00:14:12,834][02255] Updated weights for policy 0, policy_version 920 (0.0012)
[2025-01-20 00:14:16,618][00404] Fps is (10 sec: 4505.5, 60 sec: 4300.8, 300 sec: 4221.0). Total num frames: 3784704. Throughput: 0: 1045.4. Samples: 944212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:14:16,624][00404] Avg episode reward: [(0, '18.792')]
[2025-01-20 00:14:21,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4232.5, 300 sec: 4207.1). Total num frames: 3805184. Throughput: 0: 1092.9. Samples: 950990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:14:21,624][00404] Avg episode reward: [(0, '18.634')]
[2025-01-20 00:14:22,270][02255] Updated weights for policy 0, policy_version 930 (0.0011)
[2025-01-20 00:14:26,617][00404] Fps is (10 sec: 3686.5, 60 sec: 4096.0, 300 sec: 4179.3). Total num frames: 3821568. Throughput: 0: 1038.3. Samples: 955966. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2025-01-20 00:14:26,622][00404] Avg episode reward: [(0, '18.358')]
[2025-01-20 00:14:31,591][02255] Updated weights for policy 0, policy_version 940 (0.0022)
[2025-01-20 00:14:31,617][00404] Fps is (10 sec: 4505.6, 60 sec: 4300.8, 300 sec: 4221.0). Total num frames: 3850240. Throughput: 0: 1038.9. Samples: 959772. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:31,620][00404] Avg episode reward: [(0, '17.742')]
[2025-01-20 00:14:36,617][00404] Fps is (10 sec: 4915.2, 60 sec: 4300.8, 300 sec: 4221.0). Total num frames: 3870720. Throughput: 0: 1094.5. Samples: 967406. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2025-01-20 00:14:36,623][00404] Avg episode reward: [(0, '16.556')]
[2025-01-20 00:14:41,619][00404] Fps is (10 sec: 3685.7, 60 sec: 4164.1, 300 sec: 4193.2). Total num frames: 3887104. Throughput: 0: 1057.2. Samples: 972366. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:41,621][00404] Avg episode reward: [(0, '16.560')]
[2025-01-20 00:14:42,152][02255] Updated weights for policy 0, policy_version 950 (0.0015)
[2025-01-20 00:14:46,618][00404] Fps is (10 sec: 4096.0, 60 sec: 4232.5, 300 sec: 4207.1). Total num frames: 3911680. Throughput: 0: 1044.8. Samples: 975592. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:46,620][00404] Avg episode reward: [(0, '17.244')]
[2025-01-20 00:14:50,351][02255] Updated weights for policy 0, policy_version 960 (0.0015)
[2025-01-20 00:14:51,617][00404] Fps is (10 sec: 4916.1, 60 sec: 4369.1, 300 sec: 4234.8). Total num frames: 3936256. Throughput: 0: 1072.2. Samples: 983012. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:51,619][00404] Avg episode reward: [(0, '19.402')]
[2025-01-20 00:14:51,622][02242] Saving new best policy, reward=19.402!
[2025-01-20 00:14:56,618][00404] Fps is (10 sec: 4095.9, 60 sec: 4232.5, 300 sec: 4207.1). Total num frames: 3952640. Throughput: 0: 1069.2. Samples: 988646. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:14:56,626][00404] Avg episode reward: [(0, '18.696')]
[2025-01-20 00:15:01,069][02255] Updated weights for policy 0, policy_version 970 (0.0020)
[2025-01-20 00:15:01,617][00404] Fps is (10 sec: 3686.4, 60 sec: 4164.3, 300 sec: 4193.2). Total num frames: 3973120. Throughput: 0: 1042.3. Samples: 991114. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:15:01,624][00404] Avg episode reward: [(0, '18.544')]
[2025-01-20 00:15:06,617][00404] Fps is (10 sec: 4505.8, 60 sec: 4300.8, 300 sec: 4234.9). Total num frames: 3997696. Throughput: 0: 1059.9. Samples: 998684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2025-01-20 00:15:06,623][00404] Avg episode reward: [(0, '18.395')]
[2025-01-20 00:15:07,584][02242] Stopping Batcher_0...
[2025-01-20 00:15:07,585][02242] Loop batcher_evt_loop terminating...
[2025-01-20 00:15:07,587][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2025-01-20 00:15:07,586][00404] Component Batcher_0 stopped!
[2025-01-20 00:15:07,653][02255] Weights refcount: 2 0
[2025-01-20 00:15:07,662][00404] Component InferenceWorker_p0-w0 stopped!
[2025-01-20 00:15:07,667][02255] Stopping InferenceWorker_p0-w0...
[2025-01-20 00:15:07,668][02255] Loop inference_proc0-0_evt_loop terminating...
[2025-01-20 00:15:07,732][02242] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000790_3235840.pth
[2025-01-20 00:15:07,753][02242] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2025-01-20 00:15:07,940][02242] Stopping LearnerWorker_p0...
[2025-01-20 00:15:07,941][02242] Loop learner_proc0_evt_loop terminating...
[2025-01-20 00:15:07,940][00404] Component LearnerWorker_p0 stopped!
[2025-01-20 00:15:08,000][00404] Component RolloutWorker_w7 stopped!
[2025-01-20 00:15:08,007][02263] Stopping RolloutWorker_w7...
[2025-01-20 00:15:08,014][02263] Loop rollout_proc7_evt_loop terminating...
[2025-01-20 00:15:08,020][00404] Component RolloutWorker_w3 stopped!
[2025-01-20 00:15:08,025][02259] Stopping RolloutWorker_w3...
[2025-01-20 00:15:08,031][00404] Component RolloutWorker_w5 stopped!
[2025-01-20 00:15:08,038][02261] Stopping RolloutWorker_w5...
[2025-01-20 00:15:08,038][02261] Loop rollout_proc5_evt_loop terminating...
[2025-01-20 00:15:08,040][02259] Loop rollout_proc3_evt_loop terminating...
[2025-01-20 00:15:08,077][00404] Component RolloutWorker_w1 stopped!
[2025-01-20 00:15:08,083][02257] Stopping RolloutWorker_w1...
[2025-01-20 00:15:08,085][02257] Loop rollout_proc1_evt_loop terminating...
[2025-01-20 00:15:08,102][02260] Stopping RolloutWorker_w4...
[2025-01-20 00:15:08,102][00404] Component RolloutWorker_w4 stopped!
[2025-01-20 00:15:08,102][02260] Loop rollout_proc4_evt_loop terminating...
[2025-01-20 00:15:08,114][00404] Component RolloutWorker_w0 stopped!
[2025-01-20 00:15:08,115][02256] Stopping RolloutWorker_w0...
[2025-01-20 00:15:08,120][02256] Loop rollout_proc0_evt_loop terminating...
[2025-01-20 00:15:08,183][00404] Component RolloutWorker_w2 stopped!
[2025-01-20 00:15:08,185][02258] Stopping RolloutWorker_w2...
[2025-01-20 00:15:08,186][02258] Loop rollout_proc2_evt_loop terminating...
[2025-01-20 00:15:08,209][00404] Component RolloutWorker_w6 stopped!
[2025-01-20 00:15:08,211][00404] Waiting for process learner_proc0 to stop...
[2025-01-20 00:15:08,217][02262] Stopping RolloutWorker_w6...
[2025-01-20 00:15:08,217][02262] Loop rollout_proc6_evt_loop terminating...
[2025-01-20 00:15:09,842][00404] Waiting for process inference_proc0-0 to join...
[2025-01-20 00:15:09,847][00404] Waiting for process rollout_proc0 to join...
[2025-01-20 00:15:12,808][00404] Waiting for process rollout_proc1 to join...
[2025-01-20 00:15:12,811][00404] Waiting for process rollout_proc2 to join...
[2025-01-20 00:15:12,816][00404] Waiting for process rollout_proc3 to join...
[2025-01-20 00:15:12,819][00404] Waiting for process rollout_proc4 to join...
[2025-01-20 00:15:12,824][00404] Waiting for process rollout_proc5 to join...
[2025-01-20 00:15:12,827][00404] Waiting for process rollout_proc6 to join...
[2025-01-20 00:15:12,831][00404] Waiting for process rollout_proc7 to join...
[2025-01-20 00:15:12,833][00404] Batcher 0 profile tree view:
batching: 26.1789, releasing_batches: 0.0238
[2025-01-20 00:15:12,834][00404] InferenceWorker_p0-w0 profile tree view:
wait_policy: 0.0000
  wait_policy_total: 421.4425
update_model: 7.6228
  weight_update: 0.0020
one_step: 0.0057
  handle_policy_step: 525.3267
    deserialize: 13.3654, stack: 2.7720, obs_to_device_normalize: 113.3280, forward: 267.4485, send_messages: 25.9394
    prepare_outputs: 79.9980
      to_cpu: 50.4437
[2025-01-20 00:15:12,836][00404] Learner 0 profile tree view:
misc: 0.0034, prepare_batch: 14.3452
train: 74.5581
  epoch_init: 0.0047, minibatch_init: 0.0085, losses_postprocess: 0.6402, kl_divergence: 0.5443, after_optimizer: 33.7382
  calculate_losses: 27.3329
    losses_init: 0.0063, forward_head: 1.4109, bptt_initial: 18.5172, tail: 0.9363, advantages_returns: 0.3037, losses: 4.0403
    bptt: 1.8390
      bptt_forward_core: 1.7443
  update: 11.6768
    clip: 0.8577
[2025-01-20 00:15:12,838][00404] RolloutWorker_w0 profile tree view:
wait_for_trajectories: 0.2335, enqueue_policy_requests: 105.1436, env_step: 776.2342, overhead: 10.8676, complete_rollouts: 6.2605
save_policy_outputs: 18.1959
  split_output_tensors: 6.9371
[2025-01-20 00:15:12,839][00404] RolloutWorker_w7 profile tree view:
wait_for_trajectories: 0.2952, enqueue_policy_requests: 108.8546, env_step: 768.0927, overhead: 11.1998, complete_rollouts: 6.8800
save_policy_outputs: 18.0454
  split_output_tensors: 6.7644
[2025-01-20 00:15:12,840][00404] Loop Runner_EvtLoop terminating...
[2025-01-20 00:15:12,842][00404] Runner profile tree view:
main_loop: 1016.6971
[2025-01-20 00:15:12,843][00404] Collected {0: 4005888}, FPS: 3940.1
[2025-01-20 00:15:13,238][00404] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2025-01-20 00:15:13,240][00404] Overriding arg 'num_workers' with value 1 passed from command line
[2025-01-20 00:15:13,242][00404] Adding new argument 'no_render'=True that is not in the saved config file!
[2025-01-20 00:15:13,244][00404] Adding new argument 'save_video'=True that is not in the saved config file!
[2025-01-20 00:15:13,247][00404] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2025-01-20 00:15:13,248][00404] Adding new argument 'video_name'=None that is not in the saved config file!
[2025-01-20 00:15:13,249][00404] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
[2025-01-20 00:15:13,251][00404] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2025-01-20 00:15:13,252][00404] Adding new argument 'push_to_hub'=False that is not in the saved config file!
[2025-01-20 00:15:13,253][00404] Adding new argument 'hf_repository'=None that is not in the saved config file!
[2025-01-20 00:15:13,254][00404] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2025-01-20 00:15:13,255][00404] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2025-01-20 00:15:13,256][00404] Adding new argument 'train_script'=None that is not in the saved config file!
[2025-01-20 00:15:13,257][00404] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2025-01-20 00:15:13,258][00404] Using frameskip 1 and render_action_repeat=4 for evaluation
[2025-01-20 00:15:13,302][00404] Doom resolution: 160x120, resize resolution: (128, 72)
[2025-01-20 00:15:13,306][00404] RunningMeanStd input shape: (3, 72, 128)
[2025-01-20 00:15:13,308][00404] RunningMeanStd input shape: (1,)
[2025-01-20 00:15:13,325][00404] ConvEncoder: input_channels=3
[2025-01-20 00:15:13,448][00404] Conv encoder output size: 512
[2025-01-20 00:15:13,451][00404] Policy head output size: 512
[2025-01-20 00:15:13,720][00404] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2025-01-20 00:15:14,517][00404] Num frames 100...
[2025-01-20 00:15:14,647][00404] Num frames 200...
[2025-01-20 00:15:14,774][00404] Num frames 300...
[2025-01-20 00:15:14,902][00404] Num frames 400...
[2025-01-20 00:15:15,039][00404] Num frames 500...
[2025-01-20 00:15:15,171][00404] Num frames 600...
[2025-01-20 00:15:15,302][00404] Num frames 700...
[2025-01-20 00:15:15,439][00404] Num frames 800...
[2025-01-20 00:15:15,567][00404] Num frames 900...
[2025-01-20 00:15:15,696][00404] Num frames 1000...
[2025-01-20 00:15:15,824][00404] Num frames 1100...
[2025-01-20 00:15:15,954][00404] Num frames 1200...
[2025-01-20 00:15:16,092][00404] Num frames 1300...
[2025-01-20 00:15:16,235][00404] Num frames 1400...
[2025-01-20 00:15:16,375][00404] Num frames 1500...
[2025-01-20 00:15:16,505][00404] Num frames 1600...
[2025-01-20 00:15:16,633][00404] Num frames 1700...
[2025-01-20 00:15:16,762][00404] Num frames 1800...
[2025-01-20 00:15:16,890][00404] Num frames 1900...
[2025-01-20 00:15:17,066][00404] Avg episode rewards: #0: 48.839, true rewards: #0: 19.840
[2025-01-20 00:15:17,068][00404] Avg episode reward: 48.839, avg true_objective: 19.840
[2025-01-20 00:15:17,091][00404] Num frames 2000...
[2025-01-20 00:15:17,220][00404] Num frames 2100...
[2025-01-20 00:15:17,351][00404] Num frames 2200...
[2025-01-20 00:15:17,486][00404] Num frames 2300...
[2025-01-20 00:15:17,618][00404] Num frames 2400...
[2025-01-20 00:15:17,748][00404] Num frames 2500...
[2025-01-20 00:15:17,877][00404] Num frames 2600...
[2025-01-20 00:15:18,006][00404] Num frames 2700...
[2025-01-20 00:15:18,144][00404] Num frames 2800...
[2025-01-20 00:15:18,278][00404] Num frames 2900...
[2025-01-20 00:15:18,420][00404] Num frames 3000...
[2025-01-20 00:15:18,517][00404] Avg episode rewards: #0: 35.630, true rewards: #0: 15.130
[2025-01-20 00:15:18,520][00404] Avg episode reward: 35.630, avg true_objective: 15.130
[2025-01-20 00:15:18,624][00404] Num frames 3100...
[2025-01-20 00:15:18,756][00404] Num frames 3200...
[2025-01-20 00:15:18,903][00404] Num frames 3300...
[2025-01-20 00:15:19,047][00404] Num frames 3400...
[2025-01-20 00:15:19,127][00404] Avg episode rewards: #0: 25.033, true rewards: #0: 11.367
[2025-01-20 00:15:19,129][00404] Avg episode reward: 25.033, avg true_objective: 11.367
[2025-01-20 00:15:19,263][00404] Num frames 3500...
[2025-01-20 00:15:19,407][00404] Num frames 3600...
[2025-01-20 00:15:19,547][00404] Num frames 3700...
[2025-01-20 00:15:19,687][00404] Num frames 3800...
[2025-01-20 00:15:19,828][00404] Num frames 3900...
[2025-01-20 00:15:19,956][00404] Num frames 4000...
[2025-01-20 00:15:20,101][00404] Num frames 4100...
[2025-01-20 00:15:20,237][00404] Num frames 4200...
[2025-01-20 00:15:20,374][00404] Num frames 4300...
[2025-01-20 00:15:20,508][00404] Num frames 4400...
[2025-01-20 00:15:20,696][00404] Avg episode rewards: #0: 24.747, true rewards: #0: 11.247
[2025-01-20 00:15:20,699][00404] Avg episode reward: 24.747, avg true_objective: 11.247
[2025-01-20 00:15:20,704][00404] Num frames 4500...
[2025-01-20 00:15:20,832][00404] Num frames 4600...
[2025-01-20 00:15:20,984][00404] Num frames 4700...
[2025-01-20 00:15:21,128][00404] Num frames 4800...
[2025-01-20 00:15:21,263][00404] Num frames 4900...
[2025-01-20 00:15:21,403][00404] Num frames 5000...
[2025-01-20 00:15:21,532][00404] Num frames 5100...
[2025-01-20 00:15:21,662][00404] Num frames 5200...
[2025-01-20 00:15:21,794][00404] Num frames 5300...
[2025-01-20 00:15:21,895][00404] Avg episode rewards: #0: 23.662, true rewards: #0: 10.662
[2025-01-20 00:15:21,897][00404] Avg episode reward: 23.662, avg true_objective: 10.662
[2025-01-20 00:15:21,989][00404] Num frames 5400...
[2025-01-20 00:15:22,117][00404] Num frames 5500...
[2025-01-20 00:15:22,275][00404] Num frames 5600...
[2025-01-20 00:15:22,421][00404] Num frames 5700...
[2025-01-20 00:15:22,552][00404] Num frames 5800...
[2025-01-20 00:15:22,682][00404] Num frames 5900...
[2025-01-20 00:15:22,789][00404] Avg episode rewards: #0: 21.232, true rewards: #0: 9.898
[2025-01-20 00:15:22,792][00404] Avg episode reward: 21.232, avg true_objective: 9.898
[2025-01-20 00:15:22,903][00404] Num frames 6000...
[2025-01-20 00:15:23,074][00404] Num frames 6100...
[2025-01-20 00:15:23,250][00404] Num frames 6200...
[2025-01-20 00:15:23,433][00404] Num frames 6300...
[2025-01-20 00:15:23,604][00404] Num frames 6400...
[2025-01-20 00:15:23,767][00404] Num frames 6500...
[2025-01-20 00:15:23,931][00404] Num frames 6600...
[2025-01-20 00:15:24,096][00404] Num frames 6700...
[2025-01-20 00:15:24,280][00404] Num frames 6800...
[2025-01-20 00:15:24,345][00404] Avg episode rewards: #0: 20.290, true rewards: #0: 9.719
[2025-01-20 00:15:24,347][00404] Avg episode reward: 20.290, avg true_objective: 9.719
[2025-01-20 00:15:24,521][00404] Num frames 6900...
[2025-01-20 00:15:24,699][00404] Num frames 7000...
[2025-01-20 00:15:24,868][00404] Num frames 7100...
[2025-01-20 00:15:25,044][00404] Num frames 7200...
[2025-01-20 00:15:25,186][00404] Num frames 7300...
[2025-01-20 00:15:25,272][00404] Avg episode rewards: #0: 18.652, true rewards: #0: 9.152
[2025-01-20 00:15:25,274][00404] Avg episode reward: 18.652, avg true_objective: 9.152
[2025-01-20 00:15:25,392][00404] Num frames 7400...
[2025-01-20 00:15:25,521][00404] Num frames 7500...
[2025-01-20 00:15:25,653][00404] Num frames 7600...
[2025-01-20 00:15:25,782][00404] Num frames 7700...
[2025-01-20 00:15:25,913][00404] Num frames 7800...
[2025-01-20 00:15:26,045][00404] Num frames 7900...
[2025-01-20 00:15:26,176][00404] Num frames 8000...
[2025-01-20 00:15:26,305][00404] Num frames 8100...
[2025-01-20 00:15:26,446][00404] Num frames 8200...
[2025-01-20 00:15:26,574][00404] Num frames 8300...
[2025-01-20 00:15:26,703][00404] Num frames 8400...
[2025-01-20 00:15:26,830][00404] Num frames 8500...
[2025-01-20 00:15:26,896][00404] Avg episode rewards: #0: 19.451, true rewards: #0: 9.451
[2025-01-20 00:15:26,898][00404] Avg episode reward: 19.451, avg true_objective: 9.451
[2025-01-20 00:15:27,021][00404] Num frames 8600...
[2025-01-20 00:15:27,154][00404] Num frames 8700...
[2025-01-20 00:15:27,287][00404] Num frames 8800...
[2025-01-20 00:15:27,432][00404] Num frames 8900...
[2025-01-20 00:15:27,563][00404] Num frames 9000...
[2025-01-20 00:15:27,696][00404] Num frames 9100...
[2025-01-20 00:15:27,831][00404] Num frames 9200...
[2025-01-20 00:15:27,893][00404] Avg episode rewards: #0: 18.604, true rewards: #0: 9.204
[2025-01-20 00:15:27,894][00404] Avg episode reward: 18.604, avg true_objective: 9.204
[2025-01-20 00:16:18,287][00404] Replay video saved to /content/train_dir/default_experiment/replay.mp4!
[2025-01-20 00:16:18,785][00404] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2025-01-20 00:16:18,787][00404] Overriding arg 'num_workers' with value 1 passed from command line
[2025-01-20 00:16:18,789][00404] Adding new argument 'no_render'=True that is not in the saved config file!
[2025-01-20 00:16:18,790][00404] Adding new argument 'save_video'=True that is not in the saved config file!
[2025-01-20 00:16:18,792][00404] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2025-01-20 00:16:18,794][00404] Adding new argument 'video_name'=None that is not in the saved config file!
[2025-01-20 00:16:18,795][00404] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
[2025-01-20 00:16:18,797][00404] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2025-01-20 00:16:18,798][00404] Adding new argument 'push_to_hub'=True that is not in the saved config file!
[2025-01-20 00:16:18,799][00404] Adding new argument 'hf_repository'='ThomasSimonini/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
[2025-01-20 00:16:18,800][00404] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2025-01-20 00:16:18,801][00404] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2025-01-20 00:16:18,802][00404] Adding new argument 'train_script'=None that is not in the saved config file!
[2025-01-20 00:16:18,803][00404] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2025-01-20 00:16:18,804][00404] Using frameskip 1 and render_action_repeat=4 for evaluation
[2025-01-20 00:16:18,844][00404] RunningMeanStd input shape: (3, 72, 128)
[2025-01-20 00:16:18,848][00404] RunningMeanStd input shape: (1,)
[2025-01-20 00:16:18,862][00404] ConvEncoder: input_channels=3
[2025-01-20 00:16:18,911][00404] Conv encoder output size: 512
[2025-01-20 00:16:18,913][00404] Policy head output size: 512
[2025-01-20 00:16:18,939][00404] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2025-01-20 00:16:19,624][00404] Num frames 100...
[2025-01-20 00:16:19,807][00404] Num frames 200...
[2025-01-20 00:16:19,981][00404] Num frames 300...
[2025-01-20 00:16:20,146][00404] Num frames 400...
[2025-01-20 00:16:20,336][00404] Num frames 500...
[2025-01-20 00:16:20,545][00404] Num frames 600...
[2025-01-20 00:16:20,720][00404] Num frames 700...
[2025-01-20 00:16:20,925][00404] Num frames 800...
[2025-01-20 00:16:21,125][00404] Num frames 900...
[2025-01-20 00:16:21,309][00404] Num frames 1000...
[2025-01-20 00:16:21,489][00404] Num frames 1100...
[2025-01-20 00:16:21,653][00404] Num frames 1200...
[2025-01-20 00:16:21,861][00404] Num frames 1300...
[2025-01-20 00:16:22,077][00404] Num frames 1400...
[2025-01-20 00:16:22,286][00404] Num frames 1500...
[2025-01-20 00:16:22,420][00404] Avg episode rewards: #0: 42.399, true rewards: #0: 15.400
[2025-01-20 00:16:22,423][00404] Avg episode reward: 42.399, avg true_objective: 15.400
[2025-01-20 00:16:22,566][00404] Num frames 1600...
[2025-01-20 00:16:22,754][00404] Num frames 1700...
[2025-01-20 00:16:22,946][00404] Num frames 1800...
[2025-01-20 00:16:23,134][00404] Num frames 1900...
[2025-01-20 00:16:23,321][00404] Num frames 2000...
[2025-01-20 00:16:23,550][00404] Num frames 2100...
[2025-01-20 00:16:23,752][00404] Num frames 2200...
[2025-01-20 00:16:23,928][00404] Num frames 2300...
[2025-01-20 00:16:24,116][00404] Num frames 2400...
[2025-01-20 00:16:24,314][00404] Num frames 2500...
[2025-01-20 00:16:24,529][00404] Num frames 2600...
[2025-01-20 00:16:24,740][00404] Num frames 2700...
[2025-01-20 00:16:24,850][00404] Avg episode rewards: #0: 35.620, true rewards: #0: 13.620
[2025-01-20 00:16:24,852][00404] Avg episode reward: 35.620, avg true_objective: 13.620
[2025-01-20 00:16:25,012][00404] Num frames 2800...
[2025-01-20 00:16:25,235][00404] Num frames 2900...
[2025-01-20 00:16:25,455][00404] Num frames 3000...
[2025-01-20 00:16:25,664][00404] Num frames 3100...
[2025-01-20 00:16:25,877][00404] Num frames 3200...
[2025-01-20 00:16:25,929][00404] Avg episode rewards: #0: 26.000, true rewards: #0: 10.667
[2025-01-20 00:16:25,931][00404] Avg episode reward: 26.000, avg true_objective: 10.667
[2025-01-20 00:16:26,126][00404] Num frames 3300...
[2025-01-20 00:16:26,321][00404] Num frames 3400...
[2025-01-20 00:16:26,579][00404] Avg episode rewards: #0: 20.720, true rewards: #0: 8.720
[2025-01-20 00:16:26,582][00404] Avg episode reward: 20.720, avg true_objective: 8.720
[2025-01-20 00:16:26,612][00404] Num frames 3500...
[2025-01-20 00:16:26,817][00404] Num frames 3600...
[2025-01-20 00:16:27,002][00404] Num frames 3700...
[2025-01-20 00:16:27,203][00404] Num frames 3800...
[2025-01-20 00:16:27,420][00404] Num frames 3900...
[2025-01-20 00:16:27,579][00404] Avg episode rewards: #0: 18.272, true rewards: #0: 7.872
[2025-01-20 00:16:27,582][00404] Avg episode reward: 18.272, avg true_objective: 7.872
[2025-01-20 00:16:27,730][00404] Num frames 4000...
[2025-01-20 00:16:27,988][00404] Num frames 4100...
[2025-01-20 00:16:28,241][00404] Num frames 4200...
[2025-01-20 00:16:28,431][00404] Num frames 4300...
[2025-01-20 00:16:28,637][00404] Num frames 4400...
[2025-01-20 00:16:28,835][00404] Num frames 4500...
[2025-01-20 00:16:29,006][00404] Num frames 4600...
[2025-01-20 00:16:29,187][00404] Num frames 4700...
[2025-01-20 00:16:29,370][00404] Num frames 4800...
[2025-01-20 00:16:29,545][00404] Num frames 4900...
[2025-01-20 00:16:29,742][00404] Num frames 5000...
[2025-01-20 00:16:29,877][00404] Num frames 5100...
[2025-01-20 00:16:30,001][00404] Avg episode rewards: #0: 19.758, true rewards: #0: 8.592
[2025-01-20 00:16:30,003][00404] Avg episode reward: 19.758, avg true_objective: 8.592
[2025-01-20 00:16:30,065][00404] Num frames 5200...
[2025-01-20 00:16:30,192][00404] Num frames 5300...
[2025-01-20 00:16:30,316][00404] Num frames 5400...
[2025-01-20 00:16:30,451][00404] Num frames 5500...
[2025-01-20 00:16:30,582][00404] Num frames 5600...
[2025-01-20 00:16:30,675][00404] Avg episode rewards: #0: 18.184, true rewards: #0: 8.041
[2025-01-20 00:16:30,676][00404] Avg episode reward: 18.184, avg true_objective: 8.041
[2025-01-20 00:16:30,784][00404] Num frames 5700...
[2025-01-20 00:16:30,922][00404] Num frames 5800...
[2025-01-20 00:16:31,049][00404] Num frames 5900...
[2025-01-20 00:16:31,176][00404] Num frames 6000...
[2025-01-20 00:16:31,303][00404] Num frames 6100...
[2025-01-20 00:16:31,438][00404] Num frames 6200...
[2025-01-20 00:16:31,564][00404] Num frames 6300...
[2025-01-20 00:16:31,691][00404] Num frames 6400...
[2025-01-20 00:16:31,869][00404] Avg episode rewards: #0: 17.741, true rewards: #0: 8.116
[2025-01-20 00:16:31,870][00404] Avg episode reward: 17.741, avg true_objective: 8.116
[2025-01-20 00:16:31,883][00404] Num frames 6500...
[2025-01-20 00:16:32,009][00404] Num frames 6600...
[2025-01-20 00:16:32,136][00404] Num frames 6700...
[2025-01-20 00:16:32,263][00404] Num frames 6800...
[2025-01-20 00:16:32,402][00404] Num frames 6900...
[2025-01-20 00:16:32,531][00404] Num frames 7000...
[2025-01-20 00:16:32,654][00404] Num frames 7100...
[2025-01-20 00:16:32,779][00404] Num frames 7200...
[2025-01-20 00:16:32,908][00404] Num frames 7300...
[2025-01-20 00:16:33,036][00404] Num frames 7400...
[2025-01-20 00:16:33,164][00404] Num frames 7500...
[2025-01-20 00:16:33,293][00404] Num frames 7600...
[2025-01-20 00:16:33,426][00404] Num frames 7700...
[2025-01-20 00:16:33,552][00404] Num frames 7800...
[2025-01-20 00:16:33,676][00404] Num frames 7900...
[2025-01-20 00:16:33,801][00404] Num frames 8000...
[2025-01-20 00:16:33,933][00404] Num frames 8100...
[2025-01-20 00:16:34,024][00404] Avg episode rewards: #0: 19.917, true rewards: #0: 9.028
[2025-01-20 00:16:34,025][00404] Avg episode reward: 19.917, avg true_objective: 9.028
[2025-01-20 00:16:34,122][00404] Num frames 8200...
[2025-01-20 00:16:34,250][00404] Num frames 8300...
[2025-01-20 00:16:34,389][00404] Num frames 8400...
[2025-01-20 00:16:34,519][00404] Num frames 8500...
[2025-01-20 00:16:34,646][00404] Num frames 8600...
[2025-01-20 00:16:34,773][00404] Num frames 8700...
[2025-01-20 00:16:34,910][00404] Num frames 8800...
[2025-01-20 00:16:35,047][00404] Num frames 8900...
[2025-01-20 00:16:35,178][00404] Num frames 9000...
[2025-01-20 00:16:35,308][00404] Num frames 9100...
[2025-01-20 00:16:35,445][00404] Num frames 9200...
[2025-01-20 00:16:35,577][00404] Num frames 9300...
[2025-01-20 00:16:35,704][00404] Num frames 9400...
[2025-01-20 00:16:35,830][00404] Num frames 9500...
[2025-01-20 00:16:35,963][00404] Num frames 9600...
[2025-01-20 00:16:36,091][00404] Num frames 9700...
[2025-01-20 00:16:36,220][00404] Num frames 9800...
[2025-01-20 00:16:36,352][00404] Num frames 9900...
[2025-01-20 00:16:36,486][00404] Num frames 10000...
[2025-01-20 00:16:36,616][00404] Num frames 10100...
[2025-01-20 00:16:36,745][00404] Num frames 10200...
[2025-01-20 00:16:36,834][00404] Avg episode rewards: #0: 24.025, true rewards: #0: 10.225
[2025-01-20 00:16:36,835][00404] Avg episode reward: 24.025, avg true_objective: 10.225
[2025-01-20 00:17:35,049][00404] Replay video saved to /content/train_dir/default_experiment/replay.mp4!
[2025-01-20 00:17:56,123][00404] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2025-01-20 00:17:56,126][00404] Overriding arg 'num_workers' with value 1 passed from command line
[2025-01-20 00:17:56,128][00404] Adding new argument 'no_render'=True that is not in the saved config file!
[2025-01-20 00:17:56,130][00404] Adding new argument 'save_video'=True that is not in the saved config file!
[2025-01-20 00:17:56,132][00404] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2025-01-20 00:17:56,135][00404] Adding new argument 'video_name'=None that is not in the saved config file!
[2025-01-20 00:17:56,137][00404] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
[2025-01-20 00:17:56,141][00404] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2025-01-20 00:17:56,143][00404] Adding new argument 'push_to_hub'=True that is not in the saved config file!
[2025-01-20 00:17:56,155][00404] Adding new argument 'hf_repository'='SriramSohan/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
[2025-01-20 00:17:56,156][00404] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2025-01-20 00:17:56,158][00404] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2025-01-20 00:17:56,159][00404] Adding new argument 'train_script'=None that is not in the saved config file!
[2025-01-20 00:17:56,161][00404] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2025-01-20 00:17:56,162][00404] Using frameskip 1 and render_action_repeat=4 for evaluation
[2025-01-20 00:17:56,262][00404] RunningMeanStd input shape: (3, 72, 128)
[2025-01-20 00:17:56,265][00404] RunningMeanStd input shape: (1,)
[2025-01-20 00:17:56,300][00404] ConvEncoder: input_channels=3
[2025-01-20 00:17:56,494][00404] Conv encoder output size: 512
[2025-01-20 00:17:56,504][00404] Policy head output size: 512
[2025-01-20 00:17:56,571][00404] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2025-01-20 00:17:57,426][00404] Num frames 100...
[2025-01-20 00:17:57,607][00404] Num frames 200...
[2025-01-20 00:17:57,759][00404] Num frames 300...
[2025-01-20 00:17:57,886][00404] Num frames 400...
[2025-01-20 00:17:58,014][00404] Num frames 500...
[2025-01-20 00:17:58,142][00404] Num frames 600...
[2025-01-20 00:17:58,270][00404] Num frames 700...
[2025-01-20 00:17:58,403][00404] Num frames 800...
[2025-01-20 00:17:58,456][00404] Avg episode rewards: #0: 15.000, true rewards: #0: 8.000
[2025-01-20 00:17:58,457][00404] Avg episode reward: 15.000, avg true_objective: 8.000
[2025-01-20 00:17:58,587][00404] Num frames 900...
[2025-01-20 00:17:58,726][00404] Num frames 1000...
[2025-01-20 00:17:58,852][00404] Num frames 1100...
[2025-01-20 00:17:58,981][00404] Num frames 1200...
[2025-01-20 00:17:59,057][00404] Avg episode rewards: #0: 10.580, true rewards: #0: 6.080
[2025-01-20 00:17:59,058][00404] Avg episode reward: 10.580, avg true_objective: 6.080
[2025-01-20 00:17:59,170][00404] Num frames 1300...
[2025-01-20 00:17:59,303][00404] Num frames 1400...
[2025-01-20 00:17:59,439][00404] Num frames 1500...
[2025-01-20 00:17:59,571][00404] Num frames 1600...
[2025-01-20 00:17:59,711][00404] Num frames 1700...
[2025-01-20 00:17:59,840][00404] Num frames 1800...
[2025-01-20 00:17:59,970][00404] Num frames 1900...
[2025-01-20 00:18:00,097][00404] Num frames 2000...
[2025-01-20 00:18:00,263][00404] Num frames 2100...
[2025-01-20 00:18:00,428][00404] Num frames 2200...
[2025-01-20 00:18:00,557][00404] Num frames 2300...
[2025-01-20 00:18:00,692][00404] Num frames 2400...
[2025-01-20 00:18:00,828][00404] Num frames 2500...
[2025-01-20 00:18:00,955][00404] Num frames 2600...
[2025-01-20 00:18:01,086][00404] Num frames 2700...
[2025-01-20 00:18:01,216][00404] Num frames 2800...
[2025-01-20 00:18:01,351][00404] Num frames 2900...
[2025-01-20 00:18:01,495][00404] Num frames 3000...
[2025-01-20 00:18:01,625][00404] Num frames 3100...
[2025-01-20 00:18:01,770][00404] Num frames 3200...
[2025-01-20 00:18:01,868][00404] Avg episode rewards: #0: 24.440, true rewards: #0: 10.773
[2025-01-20 00:18:01,869][00404] Avg episode reward: 24.440, avg true_objective: 10.773
[2025-01-20 00:18:01,959][00404] Num frames 3300...
[2025-01-20 00:18:02,091][00404] Num frames 3400...
[2025-01-20 00:18:02,220][00404] Num frames 3500...
[2025-01-20 00:18:02,351][00404] Num frames 3600...
[2025-01-20 00:18:02,483][00404] Num frames 3700...
[2025-01-20 00:18:02,585][00404] Avg episode rewards: #0: 20.592, true rewards: #0: 9.343
[2025-01-20 00:18:02,586][00404] Avg episode reward: 20.592, avg true_objective: 9.343
[2025-01-20 00:18:02,670][00404] Num frames 3800...
[2025-01-20 00:18:02,802][00404] Num frames 3900...
[2025-01-20 00:18:02,929][00404] Num frames 4000...
[2025-01-20 00:18:03,057][00404] Num frames 4100...
[2025-01-20 00:18:03,184][00404] Num frames 4200...
[2025-01-20 00:18:03,313][00404] Num frames 4300...
[2025-01-20 00:18:03,451][00404] Num frames 4400...
[2025-01-20 00:18:03,581][00404] Num frames 4500...
[2025-01-20 00:18:03,711][00404] Num frames 4600...
[2025-01-20 00:18:03,846][00404] Num frames 4700...
[2025-01-20 00:18:03,939][00404] Avg episode rewards: #0: 20.458, true rewards: #0: 9.458
[2025-01-20 00:18:03,940][00404] Avg episode reward: 20.458, avg true_objective: 9.458
[2025-01-20 00:18:04,031][00404] Num frames 4800...
[2025-01-20 00:18:04,161][00404] Num frames 4900...
[2025-01-20 00:18:04,294][00404] Num frames 5000...
[2025-01-20 00:18:04,439][00404] Num frames 5100...
[2025-01-20 00:18:04,576][00404] Num frames 5200...
[2025-01-20 00:18:04,707][00404] Num frames 5300...
[2025-01-20 00:18:04,844][00404] Num frames 5400...
[2025-01-20 00:18:04,974][00404] Num frames 5500...
[2025-01-20 00:18:05,152][00404] Avg episode rewards: #0: 19.988, true rewards: #0: 9.322
[2025-01-20 00:18:05,154][00404] Avg episode reward: 19.988, avg true_objective: 9.322
[2025-01-20 00:18:05,166][00404] Num frames 5600...
[2025-01-20 00:18:05,295][00404] Num frames 5700...
[2025-01-20 00:18:05,429][00404] Num frames 5800...
[2025-01-20 00:18:05,561][00404] Num frames 5900...
[2025-01-20 00:18:05,719][00404] Avg episode rewards: #0: 18.253, true rewards: #0: 8.539
[2025-01-20 00:18:05,721][00404] Avg episode reward: 18.253, avg true_objective: 8.539
[2025-01-20 00:18:05,753][00404] Num frames 6000...
[2025-01-20 00:18:05,889][00404] Num frames 6100...
[2025-01-20 00:18:06,018][00404] Num frames 6200...
[2025-01-20 00:18:06,144][00404] Num frames 6300...
[2025-01-20 00:18:06,274][00404] Num frames 6400...
[2025-01-20 00:18:06,408][00404] Num frames 6500...
[2025-01-20 00:18:06,524][00404] Avg episode rewards: #0: 17.047, true rewards: #0: 8.172
[2025-01-20 00:18:06,526][00404] Avg episode reward: 17.047, avg true_objective: 8.172
[2025-01-20 00:18:06,603][00404] Num frames 6600...
[2025-01-20 00:18:06,735][00404] Num frames 6700...
[2025-01-20 00:18:06,867][00404] Num frames 6800...
[2025-01-20 00:18:06,997][00404] Num frames 6900...
[2025-01-20 00:18:07,128][00404] Num frames 7000...
[2025-01-20 00:18:07,256][00404] Num frames 7100...
[2025-01-20 00:18:07,419][00404] Avg episode rewards: #0: 16.642, true rewards: #0: 7.976
[2025-01-20 00:18:07,421][00404] Avg episode reward: 16.642, avg true_objective: 7.976
[2025-01-20 00:18:07,455][00404] Num frames 7200...
[2025-01-20 00:18:07,590][00404] Num frames 7300...
[2025-01-20 00:18:07,734][00404] Num frames 7400...
[2025-01-20 00:18:07,913][00404] Num frames 7500...
[2025-01-20 00:18:08,086][00404] Num frames 7600...
[2025-01-20 00:18:08,252][00404] Num frames 7700...
[2025-01-20 00:18:08,474][00404] Avg episode rewards: #0: 16.086, true rewards: #0: 7.786
[2025-01-20 00:18:08,477][00404] Avg episode reward: 16.086, avg true_objective: 7.786
[2025-01-20 00:18:50,896][00404] Replay video saved to /content/train_dir/default_experiment/replay.mp4!