0

when I assign a variable X to a function (f), its output is a list. variable X changes after every time that I execute my function f in everywhere in my project.

the project is about reinforcement learning. I have trouble with recording my states in the environment. when I append visited states to a list named state_history all appended states will be changing after each run of environment function and all states convert to the last state. I write a pseudo code here so you can regenerate my problem.

# this function takes current state in dictionary format, increment it's value and returns next state 
def env_(current_state):

    nex_state=current_state['current state'] +1
    current_state['current state']=nex_state
   
    return current_state
# i run function 10 times 
state_history=[]
# initializing state
initial_state={}
initial_state['current state']=0
for i in range(10):
    nex_state=env_(initial_state)
    state_history.append(nex_state)
    initial_state=nex_state
print(state_history)

here is the result: all appended states converted to the last state.

[{'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}, {'current state': 10}]

the expected result was:

[{'current state': 1}, {'current state': 2}, {'current state': 3}, {'current state': 4}, {'current state': 5}, {'current state': 6}, {'current state': 7}, {'current state': 8}, {'current state': 9}, {'current state': 10}]
Ramin Bakhtiyari
  • 21
  • 7
  • 1
    `env_` takes an object `current_state`, modifies it, and return that object itself. So every call to `env_` is operating on the single same dictionary. Do you instead want to make a **copy** of `current_state`, modify that copy and return the copy? – slothrop Aug 03 '22 at 09:13
  • @slothrop hi, yes. – Ramin Bakhtiyari Aug 03 '22 at 09:38
  • @RaminBakthtiyari The answers here should put you on the right track: https://stackoverflow.com/questions/2465921/how-to-copy-a-dictionary-and-only-edit-the-copy – slothrop Aug 03 '22 at 09:41

1 Answers1

0

Here as next_State is changing everytime, when appending into the list, it changes with updated next_state instead of adding into the already created list, this is because, you are appending main list not the copy of it. So try appending copy of next_state. Or you can go with extend() function instead of append.

BharathMV
  • 1
  • 3