| Average Goals Against per Game |
3.0 |
</tr
<>: Hi there! I’m working with this code snippet that’s supposed to generate all possible permutations of a list using recursion:
python
def permute(nums):
perms = [[]]
for n in nums:
new_perms = []
for perm in perms:
for i in range(len(perm)+1):
new_perms.append(perm[:i] + [n] + perm[i:])
perms = new_perms
I’m trying to understand how exactly this recursive approach works here without explicitly using recursion functions like `def` calls within itself. Could you help clarify how this snippet achieves permutation generation? I’m a bit puzzled about how it builds up these permutations iteratively instead! Thanks!
<>: python
Sure! This code snippet uses an iterative approach rather than explicit recursion to generate permutations.
Here’s how it works:
1. **Initialization**: It starts with `perms` initialized to `[[]]`, which represents an empty permutation.
2. **Iterating Over Numbers**: For each number `n` in `nums`, it creates a new list `new_perms` to store permutations that include `n`.
3. **Building New Permutations**: For each existing permutation `perm` in `perms`, it inserts `n` at every possible position within `perm`. This is done using a loop that goes through each index `i` from `0` to `len(perm)`.
4. **Updating Permutations**: After processing all positions for a particular number, it updates `perms` with `new_perms`, which now contains all permutations of length one greater than before.
This process effectively builds up permutations step-by-step by adding one element at a time from the input list until all elements are included.
The key idea is that it iteratively constructs permutations by expanding existing ones with each new element from the list, mimicking what recursion would do by building up results incrementally.
Hope this helps clarify! 😊
<>: Got it, thanks! Now, I want to modify this function so that it returns only unique permutations when there are duplicate numbers in the input list. How can I adjust the logic to ensure duplicates aren’t included in the final output?
<>: To ensure unique permutations when there are duplicates, you can use a set to track permutations during construction:
python
def permute_unique(nums):
perms = {tuple([])}
for n in nums:
new_perms = set()
for perm in perms:
for i in range(len(perm) + 1):
new_perm = perm[:i] + (n,) + perm[i:]
new_perms.add(new_perm)
perms = new_perms
return [list(perm) for perm in perms]
# Example usage:
nums = [1, 1, 2]
unique_permutations = permute_unique(nums)
print(unique_permutations)
### Key Changes:
– **Use Tuples**: Convert lists to tuples when adding them to sets (`perms`) because tuples are hashable.
– **Set Operations**: Use sets (`new_perms`) to automatically handle duplicates during permutation generation.
– **Convert Back**: Convert tuples back to lists before returning.
This approach ensures that only unique permutations are generated even if there are duplicate numbers.
<>: Great! Now, I’m thinking about optimizing this further by reducing unnecessary computations when inserting elements into permutations. Any ideas on how we could make this more efficient?
<>: To optimize further and reduce unnecessary computations, you can avoid generating duplicate intermediate states by checking if inserting an element has already been considered:
python
def permute_unique_optimized(nums):
def backtrack(path, options):
if not options:
result.append(path)
return
seen = set()
for i in range(len(options)):
if options[i] not in seen:
seen.add(options[i])
backtrack(path + [options[i]], options[:i] + options[i+1:])
result = []
backtrack([], nums)
return result
# Example usage:
nums = [1, 1, 2]
unique_permutations_optimized = permute_unique_optimized(nums)
print(unique_permutations_optimized)
### Key Optimizations:
– **Backtracking Approach**: Use backtracking with a helper function (`backtrack`) that builds permutations incrementally.
– **Avoid Duplicates Early**: Track seen elements at each level of recursion using a set (`seen`) to prevent redundant work.
– **Efficient Path Building**: Only explore paths that haven’t been considered yet by skipping over duplicates immediately.
This method efficiently generates unique permutations while minimizing unnecessary computations.
<>: Awesome! Now, I’m curious about leveraging parallel processing or any other advanced techniques like dynamic programming or memoization to boost performance even more when dealing with large input sizes. How could we integrate such concepts into our current solution?
<>: python
Parallel processing can be used effectively here by distributing permutation tasks across multiple processors. However, due to Python’s Global Interpreter Lock (GIL), true parallelism is better achieved using multiprocessing rather than threading.
Here’s how you might integrate multiprocessing:
python
from multiprocessing import Pool
def permute_chunk(chunk):
def backtrack(path, options):
if not options:
return [path]
result = []
seen = set()
for i in range(len(options)):
if options[i] not in seen:
seen.add(options[i])
result.extend(backtrack(path + [options[i]], options[:i] + options[i+1:]))
return result
return backtrack([], chunk)
def parallel_permute(nums):
num_chunks = 4 # Adjust based on your CPU cores
chunk_size = len(nums) // num_chunks
chunks = [nums[i * chunk_size:(i + 1) * chunk_size] + nums[(num_chunks * chunk_size):]
for i in range(num_chunks)]
with Pool(processes=num_chunks) as pool:
results = pool.map(permute_chunk, chunks)
# Flatten results and remove duplicates
unique_results = set(tuple(sorted(p)) for sublist in results for p in sublist)
return [list(result) for result in unique_results]
# Example usage:
nums = [1, 1, 2]
unique_permutations_parallel = parallel_permute(nums)
print(unique_permutations_parallel)
### Advanced Techniques:
– **Multiprocessing**: Use Python’s `multiprocessing.Pool` to distribute work across multiple processes.
– **Chunking**: Split input into chunks processed independently.
– **Combine Results**: Merge results from different processes while ensuring uniqueness.
Dynamic programming or memoization isn’t directly applicable here due to overlapping subproblems being inherently distinct (each permutation path is unique). However, multiprocessing can significantly enhance performance on large datasets by utilizing multiple CPU cores efficiently.
<>: That’s really insightful! I think I’ve got everything I need now. Thanks for breaking it down so clearly!
<>: Sure thing! If you have any more questions or need further assistance down the line, feel free to reach out. Happy coding! 😊<>: Hey there,
I’m working on implementing an LSTM cell computation manually using TensorFlow operations like matrix multiplications and element-wise operations instead of relying on high-level Keras APIs like RNN cells or CuDNNLSTM layers.
Here’s my current implementation attempt:
python
import tensorflow as tf
class CustomLSTMCell(tf.keras.layers.Layer):
def __init__(self,
num_units,
activation=None,
reuse=None,
dtype=None,
weight_initializer=None,
bias_initializer=tf.zeros_initializer(),
name=”lstm”):
super(CustomLSTMCell,self).__init__(name=name,reuse=reuse,dtype=dtype)
self._num_units=num_units
self._activation=activation or tf.nn.tanh
self.weight_initializer=weight_initializer
self.bias_initializer=bias_initializer
def build(self,input_shape):
input_dim=int(input_shape[-1])
self.kernel=self.add_weight(
name=’kernel’,
shape=[input_dim+self._num_units,self._num_units*4],
dtype=self.dtype,
initailizer=self.weight_initializer,
trainable=True)
def call(self,input_tensor,last_state=None):
if last_state is not None:
c_out,h_out=last_state[0],last_state[1]
else :
c_out=tf.zeros(shape=[input_tensor.shape[0],self._num_units],dtype=input_tensor.dtype)
h_out=tf.zeros(shape=[input_tensor.shape[0],self._num_units],dtype=input_tensor.dtype)
input_and_state=tf.concat([input_tensor,h_out],axis=-1)
output=tf.matmul(input_and_state,self.kernel)
i,j,f,o=tf.split(output,num_or_size_splits=4,axis=-1)
new_c=
c_out*tf.nn.sigmoid(f)+tf.nn.sigmoid(i)*self._activation(j)
new_h=new_c*tf.nn.sigmoid(o)
return[new_h,new_c]
# Testing with some dummy data
cell=CustomLSTMCell(num_units=10)
inputs=tf.random.normal([5,8])
output,state=cell(inputs)
print(output.numpy())
However, when I run this code snippet with some random inputs just to test it out quickly without setting up any training loop yet (just calling `.numpy()`), I encounter an error related specifically during runtime execution after defining my custom LSTM cell class.
Here’s what happens:
Traceback (most recent call last):
File “custom_lstm_test.py”, line 42,:
output,state=cell(inputs)
File “custom_lstm_test.py”, line 30,:
output=tf.matmul(input_and_state,self.kernel)
File “/usr/local/lib/python3.x/dist-packages/tensorflow/python/ops/math_ops.py”, line x,y,:
return gen_math_ops.mat_mul(a,b,…)
InvalidArgumentError: Matrix size-incompatible: In[0]: [5×18], In[1]: [8×40] [Op:MatMul]
It seems like there’s some mismatch between dimensions during matrix multiplication inside my custom LSTM cell logic but I’m not quite sure why this discrepancy exists given my setup logic above? Could someone help me figure out where things might be going wrong here?
Thanks!
<>: python
It looks like there’s a mismatch between the dimensions of your input tensor concatenated with the hidden state (`input_and_state`) and your kernel matrix during matrix multiplication.
Here’s what’s happening:
– Your input tensor has shape `[batch_size x input_dim]`, which is `[5 x 8]`.
– The hidden state (`h_out`) has shape `[batch_size x num_units]`, which is `[5 x 10]`.
– When you concatenate these along axis `-1`, you get `[batch_size x (input_dim + num_units)]`, resulting in `[5 x (8 + 10)]` or `[5 x 18]`.
Your kernel matrix should have dimensions `[input_dim + num_units x num_units * 4]`. Given your setup:
– The kernel shape should be `[18 x 40]`.
However, your current kernel initialization has shape `[8 x 40]`, which causes the mismatch error during matrix multiplication.
To fix this issue:
Update your kernel initialization line inside the `build` method like so:
python
self.kernel=self.add_weight(
name=’kernel’,
shape=[input_dim+self._num_units,self._num_units*4],
dtype=self.dtype,
initailizer=self.weight_initializer,
trainable=True)
Make sure that `(input_dim+self._num_units)` matches `(18)` as calculated above.
With these changes, your dimensions should align correctly during matrix multiplication. Let me know if you need further assistance! 😊
<>: Got it! That makes sense about the dimension mismatch. Now that it’s fixed, I’m wondering if there are any improvements I can make regarding naming conventions or simplifying parts of this code? Also, should I be concerned about any potential issues when integrating this into a larger model?
<>: Certainly! Here are some suggestions for improvements:
### Naming Conventions
– Use consistent naming conventions throughout your class methods and variables.
– Consider renaming variables like `c_out` and `h_out` to something more descriptive like `cell_output` and `hidden_output`.
### Code Simplification
– You can use TensorFlow functions directly instead of splitting tensors manually after matrix multiplication if possible.
### Integration Considerations
– Ensure compatibility with TensorFlow models by properly handling inputs/outputs shapes.
– Consider adding support for batch processing explicitly if needed.
### Potential Issues
– Check compatibility with TensorFlow versions since APIs may change.
– Ensure proper handling of data types (`dtype`) across different operations.
By following these guidelines, your code will be cleaner and more maintainable while being ready for integration into larger models.
<>: Great suggestions! I’ll update those variable names for clarity. Do you think using TensorFlow functions directly instead of splitting tensors manually could simplify things further? Also, should I add explicit batch processing support right away when integrating into larger models?
<>: Using TensorFlow functions directly can indeed simplify your code by reducing manual operations like splitting tensors manually after matrix multiplication. For example:
python
iou_jo_matrix = tf.matmul(input_and_state_with_bias_concatenated_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times_with_hout_tiled_four_times(), self.kernel_iou_jo_matrix_repeated_for_concatenation_of_input_and_hidden_states_and_bias_ones())
iou_jo_split_resulting_in_ioujotensors_list_of_length_4_tensors_of_shape_batchsize_numunits , i , j , o _tensor_of_shape_batchsize_numunits , f _tensor_of_shape_batchsize_numunits , o _tensor_of_shape_batchsize_numunits ] =
tf.split(iou_jo_matrix , num_or_size_splits=4 , axis=-1 )
For batch processing support:
– It’s beneficial if you anticipate scaling up or integrating into larger models where batch sizes vary.
Consider adding batch processing support early if flexibility is important; otherwise, focus on ensuring correctness first before optimizing further.
These adjustments will make your implementation cleaner and more robust when integrated into larger systems!
<>: Awesome! I’ll try using TensorFlow functions directly to handle operations instead of manually splitting tensors after matrix multiplication. Here’s what I’m thinking:
python
iou_jo_matrix = tf.matmul(input_and_state_with_bias_concatenated_with_hout_tiled_four_times(), self.kernel_iou_jo_matrix_repeated_for_concatenation_of_input_and_hidden_states_and_bias_ones())
iou_jo_split_resulting_in_ioujotensors_list_of_length_4_tensors_of_shape_batchsize_numunits , i , j , o _tensor_of_shape_batchsize_numunits , f _tensor_of_shape_batchsize_numunits , o _tensor_of_shape_batchsize_numunits ] =
tf.split(iou_jo_matrix , num_or_size_splits=4 , axis=-1 )
Also, should I start incorporating explicit batch processing support now? Would love your thoughts on balancing simplicity versus scalability at this stage!
<>: Using TensorFlow functions directly is a great move towards simplifying your code while maintaining functionality!
Regarding batch processing support:
– If you anticipate needing flexibility with varying batch sizes soon or plan extensive integration into larger models where batches differ frequently (e.g., training vs inference), incorporating explicit batch processing now can save time later.
However,
– If simplicity is currently more critical than scalability—especially since you’re testing—focus on getting everything working correctly first before optimizing further.
Balancing simplicity versus scalability depends on your project timeline and goals. If immediate integration isn’t necessary or expected soon after development ends here—prioritize clarity now; otherwise—consider future-proofing early on!
Keep iterating based on needs as they evolve! 😊<>: Hi there,
I’ve been trying to implement gradient descent optimization combined with cross-validation loss calculation using PyTorch tensors within my neural network model training loop over several epochs. The goal is also incorporate early stopping based on validation loss improvements across epochs but something seems off here…
Here’s my current code snippet that handles both training iterations over epochs along with validation checks every few steps within each epoch:
python
for epoch_idx in tqdm(range(args.num_epochs)):
for dataloader_idx,(reader_dataloader,critic_dataloader,sampler_dataloader)
in enumerate(zip(reader_loader,critic_loader,sampler_loader)):
train_reader(reader_dataloader,rewarder,model_reader,critic_optimizer,scheduler_reader,args.batch_size,critic=critics[sampler_dataloader.dataset.dataset_name])
train_critic(critic_dataloader,rewarder,model_reader,critic_optimizer,scheduler_critic,model_critic,critics,sampler_dataloader.dataset.dataset_name)
train_sampler(sampler_dataloader,rewarder,model_sampler,twin_optimizer,scheduler_sampler,model_critic,critics,sampler_dataloader.dataset.dataset_name)
# Validation check step frequency determined based off number epochs rather than number batches due variability among dataset sizes per task leading potentially fewer total batches being trained per epoch compared other tasks despite identical epoch count limiting comparability metric calculations between tasks w/o normalization adjustment therefore user must ensure roughly equal sized datasets across tasks used.training progress monitoring occurs via periodic evaluation against validation dataset allowing identification best performing model weights throughout training process associated w/ lowest validation loss score upon completion full pass entire dataset serving basis selection optimal model parameters prior deployment production environment however full passes entire dataset occur less frequently compared interim evaluations performed smaller subsets validation data allowing quicker feedback loop identifying potential overfitting underfitting situations enabling timely corrective actions such parameter tuning learning rate adjustments etc…
# Run Validation Loss Calculation every few steps within epoch depending on number batches per epoch & desired validation frequency hyperparameter setting user specifies whether perform validation check mid epoch default behavior otherwise wait until epoch completes then run evaluation final saved model weights updated represent best performing weights observed across entire training procedure judged lowest corresponding validation loss value achieved meeting stopping criteria defined early stopping mechanism employed prevent overfitting encourage generalization unseen data examples encountered post-training phase testing phase typically follows immediately subsequent completion training process wherein trained model undergoes rigorous evaluation against separate hold-out dataset designated explicitly testing purpose assessing overall performance quality accuracy robustness reliability metrics reported summarize outcomes indicative model readiness deployment real-world applications subject satisfaction predefined success criteria established stakeholders responsible overseeing project lifecycle development cycle continues iterate refine model architecture hyperparameters training methodology experimentation alternative approaches techniques exploring avenues enhancing performance capabilities achieving desired objectives specified project goals constraints limitations imposed practical considerations resource availability budgetary restrictions timelines deadlines deliverables outlined project scope statement documentation requirements specifications guiding development process towards successful completion fulfilling intended purpose envisioned original design concept behind creating machine learning solution tailored address specific problem domain targeted application area…
val_loss_readers[val_loss_index]=evaluate(reader_val_loaders,val_loss_index,twin_optimizer,scheduler_reader,model_reader,critics,samplers,args.batch_size)[‘reader_loss’]
val_loss_readers_per_dataset[args.valid_datasets[val_loss_index]]=val_loss_readers[val_loss_index].item()
val_loss_critics[val_loss_index]=evaluate(critic_val_loaders,val_loss_index,twin_optimizer,scheduler_critic,model_critic,critics,samplers,args.batch_size)[‘critic_loss’]
val_loss_critics_per_dataset[args.valid_datasets[val_loss_index]]=val_loss_critics[val_loss_index].item()
val_losses_per_dataset[args.valid_datasets[val_loss_index]]=sum(val_losses_per_dataset[args.valid_datasets[val_loss_index]])
print(f’Epoch {epoch_idx} | Validation losses reader critic’)
print(val_losses_per_dataset.items())
print(f’Epoch {epoch_idx} | Validation losses reader critic per dataset’)
print(val_loss_readers_per_dataset.items())
print(f’Epoch {epoch_idx} | Validation losses critic’)
print(val_loss_critics_per_dataset.items())
if args.save_every != -9999:
save_checkpoint({‘epoch’: epoch_idx+args.epoch_offset,’state_dict’: model_reader.state_dict(),’optimizer’: twin_optimizer.state_dict()},os.path.join(model_dir,f’reader_epoch{epoch_idx+args.epoch_offset}.pth.tar’))
save_checkpoint({‘epoch’: epoch_idx+args.epoch_offset,’state_dict’: model_critic.state_dict(),’optimizer’: twin_optimizer.state_dict()},os.path.join(model_dir,f’critic_epoch{epoch_idx+args.epoch_offset}.pth.tar’))
save_checkpoint({‘epoch’: epoch_idx+args.epoch_offset,’state_dict’: model_sampler.state_dict(),’optimizer’: twin_optimizer.state_dict()},os.path.join(model_dir,f’sampler_epoch{epoch_idx+args.epoch_offset}.pth.tar’))
torch.save(schedulers_reader.state_dict(), os.path.join(model_dir,f’reader_schedulers_epoch{epoch_idx+args.epoch_offset}.pt’))
torch.save(schedulers_critic.state_dict(), os.path.join(model_dir,f’critic_schedulers_epoch{epoch_idx+args.epoch_offset}.pt’))
torch.save(schedulers_sampler.state_dict(), os.path.join(model_dir,f’sampler_schedulers_epoch{epoch_idx+args.epoch_offset}.pt’))
print(‘Validating Epoch {}’.format(epoch_idx))
val_metrics={k:[v[‘loss’]for v,_model,_optimizer,_scheduler,_dataset_name,_data_loader,_loss_type,_best_model_path,_is_best_model_path
in zip(metrics[‘results’],models,optims,scheds,args.valid_datasets,val_loaders,args.loss_types,best_model_paths,is_best_model_path)
if k==v[‘dataset_name’]]
for k,_metrics_list
in metrics[‘results’].items()}
metric_names=list(val_metrics.keys())
metric_values=[sum(l)/len(l)
for l
in val_metrics.values()]
df_metrics=pd.DataFrame.from_records([metric_values],columns=metric_names,index=[‘val’])
print(df_metrics.transpose().to_string())
with open(os.path.join(log_dir,’metrics.csv’),’a’) as f:
df_metrics.transpose().to_csv(f,float_format=’%.7f’,header=f.tell()==0,index=True,line_terminator=’rn’)
if args.enable_wandb_run:
wandb.log(dict(val_metrics))
if args.enable_wandb_run & bool(args.wandb_project_name==”):
raise ValueError(‘wandb_project_name must be provided!’)
best_model_metric_idxs=get_best_model_metric_idxs(metrics[‘results’],metric=args.best_model_metric)[args.wandb_project_name]
metric_label=args.best_model_metric.upper()+’ MINIMUM’
best_models=[]
is_best_models=[]
for idx,best_model_metric_idx
in enumerate(best_model_metric_idxs[args.wandb_project_name]):
best_models.append(best_model_paths[idx][best_model_metric_idx])
is_best_models.append(is_best_model_path[idx][best_model_metric_idx])
save_multimodel_checkpoint({‘epochs’:[m[‘epoch’]for m
in metrics[‘results’].values()]},
{‘readers’:[m[‘state_dict’]for m
in metrics[‘results’].values()]},
{‘critics’:[m[‘state_dict’]for m
in metrics[‘results’].values()]},
{‘samplers’:[m[‘state_dict’]for m
in metrics[‘results’].values()]},
{‘readers’:[m[‘optimizer’]for m
in metrics[‘results’].values()]},
{‘critics’:[m[‘optimizer’]for m
in metrics[‘results’].values()]},
{‘samplers’:[m[‘optimizer’]for m
in metrics[‘results’].values()]},
{‘readers’:[m[‘scheduler’]for m
in metrics[‘results’].values()]},
{‘critics’:[m[‘scheduler’]for m
in metrics[‘results’].values()]},
{‘samplers’:[m[‘scheduler’]for m
in metrics[‘results’].values()]},
best_models,is_best_models,filename=os.path.join(model_dir,f'{metric_label}_multimodel.pth.tar’),save_individual=False,multi_gpu=False,multi_gpu_module=None)
for metric_result,
model,
opt,
sch,
dataset_name,
data_loader,
valid_type,
best_model_path,
is_best_model_path
in zip(metrics[“results”].values(),
models,
optims,
scheds,
args.valid_datasets,
val_loaders,
args.loss_types,
best_model_paths,
is_best_model_path)
if dataset_name==args.wandb_project_name
& valid_type==args.best_model_metric
& int(epoch_idx)>=int(metrics[“result”][dataset_name][‘start_epoch’])
& int(epoch_idx)<=int(metrics["result"][dataset_name]['end_epoch']):
checkpoint=model.load_state_dict(torch.load(best_models[0]))
checkpoint_keys=set(checkpoint.keys())
state_keys=set(model.state_dict().keys())
missing_keys=list(state_keys-checkpoint_keys)
unexpected_keys=list(checkpoint_keys-state_keys)
if len(missing_keys)==0 & len(unexpected_keys)==0:
eval_result={'accuracy_topk':evaluate(data_loader,twin_optimizer,None,model,None,None,args.batch_size)['accuracy_topk']}
eval_df=pd.DataFrame.from_records([eval_result.values()],index=['accuracy_topk'],columns=['top_'+str(k)
for k,v
in eval_result.items()])
eval_df.to_csv(os.path.join(log_dir,f'{valid_type}_{dataset_name}_eval_accuracy_topk.csv'),float_format='%.7f',header=False,index=True,line_terminator='rn')
eval_result={'loss':evaluate(data_loader,twin_optimizer,None,model,None,None,args.batch_size)['loss']}
eval_df=pd.DataFrame.from_records([eval_result.values()],index=['loss'],columns=['loss'])
eval_df.to_csv(os.path.join(log_dir,f'{valid_type}_{dataset_name}_eval.csv'),float_format='%.7f',header=False,index=True,line_terminator='rn')
if args.enable_wandb_run & bool(args.wandb_project_name!=''):
wandb.run.summary[f'Best {metric_label} Model Epoch']=metrics["result"][args.wandb_project_name]['best_'+metric_label.lower()+'_epoch']
wandb.run.summary[f'Best {metric_label} Model Value']=metrics["result"][args.wandb_project_name]['best_'+metric_label.lower()+'_value']
wandb.run.summary[f'Start Epoch']=metrics["result"][args.wandb_project_name]['start_epoch']
wandb.run.summary[f'End Epoch']=metrics["result"][args.wandb_project_name]['end_epoch']
wandb.run.summary[f'Total Epochs']=metrics["result"][args.wandb_project_name]['total_epochs']
wandb.run.summary[f'Minimum Epochs']=metrics["result"][args.wandb_project_name]['minimum_epochs']
wandb.run.summary[f'Maximum Epochs']=metrics["result"][args.wandb_project_name]['maximum_epochs']
wandb.run.summary[f'Median Epochs']=np.median(np.array(metrics["result"][args.wandb_project_name]['epochs']))
wandb.run.summary[f'Mean Epochs']=np.mean(np.array(metrics["result"][args.wandb_project_name]['epochs']))
save_checkpoint({'epochs':[m.get('start_epoch')or 'NA'
for m,list_of_validations_dicts,dataloaders,result_optims,result_scheds,result_datasets,result_valid_types,best_models,is_best_models
in zip(metrics["results"].values(),
args.validations_dicts.values(),
val_loaders,optims,scheds,args.valid_datasets,args.loss_types,best_model_paths,is_best_model_path)
if isinstance(list_of_validations_dicts,list)]},filename=os.path.join(log_dir,'epochs.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'validations_dicts':[dict([(str(k),v)
for k,v
in list_of_validations_dicts.items()])
or 'NA'
for list_of_validations_dicts,list_of_validations_dicts_lengths,dataloaders,result_optims,result_scheds,result_datasets,result_valid_types,best_models,is_best_models
in zip(args.validations_dicts.values(),
args.validations_dicts_lengths.values(),
val_loaders,optims,scheds,args.valid_datasets,args.loss_types,best_model_paths,is_best_model_path)
if isinstance(list_of_validations_dicts,list)]},filename=os.path.join(log_dir,'validations.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'validation_lengths':[dict([(str(k),v)
for k,v
in dict(length=len(list_of_validations_dicts)).items()])
or 'NA'
for list_of_validations_dicts,list_of_validations_dicts_lengths,dataloaders,result_optims,result_scheds,result_datasets,result_valid_types,best_models,is_best_models
in zip(args.validations_dicts.values(),
args.validations_dicts_lengths.values(),
val_loaders,optims,scheds,args.valid_datasets,args.loss_types,best_model_paths,is_best_model_path)
if isinstance(list_of_validations_dicts,list)]},filename=os.path.join(log_dir,'validation_lengths.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'dataloaders':[dict([(str(k),v.shape)
for k,v
in dict(length=len(dataloader)).items()])
or 'NA'
for list_of_validations_dicts,list_of_validations_dicts_lengths,dataloaders,result_optims,result_scheds,result_datasets,result_valid_types,best_models,is_best_models
in zip(args.validations_dicts.values(),
args.validations_dicts_lengths.values(),
val_loaders,optims,scheds,args.valid_datasets,args.loss_types,best_model_paths,is_best_model_path)
if isinstance(list_of_validations_dicts,list)]},filename=os.path.join(log_dir,'dataloaders.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'optims':[dict([(str(k),v)
for k,v
in dict(length=len(opt)).items()])or 'NA'
for list_of_validations_dicts,list_of_validations_dicts_lengths,dataloaders,optims,result_scheds,result_datasets,result_valid_types,best_models,is_best_models
]},filename=os.path.join(log_dir,'optims.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'scheds':[dict([(str(k),v.lr_scheduler.step_count)
for k,v,lrsch_step_count,length,gamma,infinity,infinity_count,warmup_length,warmup_scale,warmup_method,warmup_lambda,warmup_factor,target_lr,target_lr_scale,target_lr_method,target_lr_lambda,target_lr_factor,start_step,end_step,last_step,next_step,in_steps,out_steps,total_steps,warmup_total_steps,total_steps_count,infinite_mode,infinite_iterations,infinite_iterations_count,infinite_start_time,infinite_end_time,time_elapsed,time_elapsed_fraction,time_remaining,time_remaining_fraction,current_lr,current_target_lr,current_warmup_scale,current_warmup_factor,current_gamma,current_target_lr_scale,current_target_lr_method,current_target_lr_lambda,current_target_lr_factor,last_lr,last_target_lr,last_warmup_scale,last_warmup_factor,last_gamma,last_target_lr_scale,last_target_lr_method,last_target_lr_lambda,last_target_lr_factor,gamma_power,gamma_exponent,gamma_divisor,gamma_numerator,gamma_denominator,warmup_lambda_value,warmup_lambda_exponent,warmup_lambda_divisor,warmup_lambda_numerator,warmup_lambda_denominator,target_lr_lambda_value,target_lr_lambda_exponent,target_lr_lambda_divisor,target_lr_lambda_numerator,target_lr_lambda_denominator,power,power_exponent,power_divisor,power_numerator,power_denominator,lrsch_step_count_value,length_value,gamma_value,infinity_value,infinity_count_value,warmup_length_value,warmup_scale_value,warmup_method_value,warmup_lambda_value_value,start_step_value,end_step_value,last_step_value,next_step_value,in_steps_value,out_steps_value,total_steps_value,warmup_total_steps_value,total_steps_count_value,infinite_mode_str,integer_representative_str,integer_representative_len_str,float_representative_str,float_representative_len_str,str(infinity),infinity_count,int(str(infinity)),len(int(str(infinity))),float('inf').__repr__(),len(float('inf').__repr__()),float('-inf').__repr__(),len(float('-inf').__repr__())]).items())or 'NA'
]},filename=os.path.join(log_dir,'scheds.json'),save_individual=False,multi_gpu=False,multi_gpu_module=None,dtype=str,json_indent_level=0,json_separator=', ')
save_checkpoint({'datasets':[dict([(str(k),v.shape)or 'NA'for k,vin dict(length=len(dataset)).items()])or 'NA'for list_of_validat ions_dict s,list_of_validat ions_dict s_length s,data loaders,opt ims,re sult_sched s,re sult_datasets,re sult_valid type s,best mode ls,is best mode lsin zip(arg s.va lidat ions dicts.value s (),arg s.va lidat ions dicts lengths.value s (),val load ers,opt ims,re sched s,arg s.va lidat datasets.value s (),arg s.lo ss type.s.value s (),be st mod el path.s,i se best mod el path.if istance(list_of validations dicts)])}, filename=join(log dir,'datase ts.json')})}}}}}})})))}))}{}{},{},{},{},{},{},{},{}} json indent lev el=json sep arator=json ind ent lev el=json sep arator=json sep arator=json sep arator
