I'm wondering should we use the action_prob without multiplying legal_action to calculate the log_p or not?

Similar concern as the reward shaping case we discussed several days ago (when only the postpone is valid but model output prefers some PM).

not corresponds to the actual parameters. Absence better than errors.

Where is this used?

There doesn't seem to be a need to subclass AbsBlock. Can you try inheriting from AbsCoreModel and use PM and VM as components?

Can you make legal_action a part of states so we don't have to change the call interface? There is no restriction on what type states should be in forward.

use a tuple (states, legal_action) as states to input the function

pm_dims = [self._pm_input_dim * self._pm_num] + self._hidden_dims[:2]

why previous _pm_state_dim? (why not x.view(...))

similar question to the pm one

why design the ratio like this? what's the meaning of it?

to use the PPO algorithm

to encourage bigger entropy?

to prevent the action probability converge too fast

is it still runnable? examples.vm_scheduling.refine_rl.common seems not exist (at least in this PR)

It runnable in my environment, as I have the examples.vm_scheduling.refine_rl.common. So I don't notice this.

return buf["info"][1:] but del buf["info"][:-1]?

info store the legal_action; In DQN, exp_set should store the next_legal_action just like the next_states. But legal_action is useless in the AC training process. So I use the same treatment as the DQN.

If it is useless, why add it into buf["info"] then?

no CombineNet and SequenceNet anymore

I'll fix it

For the situation where the PM capacity varied, the calculation method is wrong.

remain_cpu = (total_pm_info[:, 0] - total_pm_info[:, 2]) / max_cpu_capacity
remain_memory = (total_pm_info[:, 1] - total_pm_info[:, 3]) / max_memory_capacity

would be better

You're right, I'll fix it

Add comment for the PM attributes used (why * 2)

I'll add a variable PM_DIM and explain the variable

keep all the history?

The history_pm_state will store the total history information. Although some history information will never be used, it's difficult to remove the numpy data, so I choose store them all.

200 or adjusted based on the duration in the config?

I'll add a new variable to replace the 200

since the total_vm_info already initialized as zeros, no need to assign zeros again

I'll remove the else condition

potential bugs.

no vm info/order checking,
if the requests info saved in self._vm_states less than the actual requests, wrong VM states info (re-starting from idx 0) would be used.

I'll add a judgement to determine whether the vm states and current simulation are matched

Since the latter 3 are actual information got by peeking.
How about adding switches in the config to decide to use them or not?

I'll add a switch to determine peeking or not

The PM_ATTRIBUTES here are not aligned with the VM_ATTRIBUTES.
The previous one is used to extract features from the snapshot_list, but the latter one is the list of features used.

I will add PM_EXTRACTED_ATTRIBUTES to represent the features extracted from the snapshot_list, and PM_ATTRIBUTES to represent the list of features used.

# get the legal pm
legal_pm = np.zeros(self._pm_num + 1)
legal_pm[self.pm_num] = 1
if len(decision_event.valid_pms) > 0:
    remain_cpu_dict = dict()
    for pm in decision_event.valid_pms:
        # if two pm has same remaining cpu, only choose the one which has smaller id
        if total_pm_info[-1, pm, 0] not in remain_cpu_dict.keys():
            remain_cpu_dict[total_pm_info[-1, pm, 0]] = 1
            legal_pm[pm] = 1

would be enough

this file not exist

this file not exist

where does this file come from?

Add comments for this file, used for what ?

Or you can add an README for this whole example, to introduce what's the files in this folder, what's the steps (generating data, get VM states, run ...)

So you implement a new Learner instead of using the original one?

why switch line here?