2

I'm trying to convert some pytorch code to tensorflow. In the pytorch code they are adding some extra parameter on every module of a model using module.register_parameter(name, param). How can i covert this part of code on tensorflow?

Sample code on below:

for module_name, module in self.model.named_modules():
    module.register_parameter(name, new_parameter)
Al Shahreyaj
  • 211
  • 1
  • 9
  • 1
    I found out that i can use `layer.add_weight()` to add a new variable to a layer. – Al Shahreyaj Jun 07 '22 at 08:34
  • Do you know how to call a registered parameter in TensorFlow, like the below code in PyTorch? Thanks `>>> module.register_parameter(name, new_parameter) >>> module.name` – Yiding Dec 31 '22 at 08:37

1 Answers1

0

tf.Variable is the equivalent of nn.Parameter in PyTorch. tf.Variable is mainly used to store model parameters since their values are constantly updated during training.

To use a tensor as a new model parameter, you need to convert it to tf.Variable. You can check here how to create variables from tensors.

If you want to add a model parameter in TensorFlow inside the model itself, you could simply create a variable inside the model class and it will be automatically registered as a model parameter by TensorFlow.

If you want to add a tf.Variable externally to a model as a model parameter, you could manually add it to the trainable_weights attribute of tf.keras.layers.Layer by extending it like this -

model.layers[-1].trainable_weights.extend([new_parameter])
Aravind G.
  • 401
  • 5
  • 13
  • Thanks for your answer. I found out that i can use `layer.add_weight()` to add a new variable to a layer. Is this equivalent to `module.register_parameter()` on pytorch? – Al Shahreyaj May 17 '22 at 10:42
  • It seems you didn't find the answer you were looking for but yet the answer is accepted. Is that correct ? – Mohan Radhakrishnan Aug 17 '23 at 08:19
  • @MohanRadhakrishnan I don't remember exactly, I worked on this problem a long time ago. But I think this answer was helpful for finding the solution, that's why I accepted it. – Aravind G. Aug 29 '23 at 11:16