I was using salt-sproxy for a while with our Juniper-based network, and all was working fine.
I started to find some problems with the new Junos code we use (18.x)
I can communicate with the device (net.connected, net.cli, grains....etc), but I can't configure anything on it via neither net.cli nor net.load_config/template, the device isn't reporting back any diff, and sees that the new configs are already configured, even if the new set file contains new configs not configured on the device for sure.
And I can see nothing coming back from the device via rpc-reply while debugging the request.
I tested that for multiple Juniper devices with the same code, and all are the same wrong behaviour.
salt-sproxy '10-220-1-9' net.load_config text='set vlans vlan888 vlan-id 888'
10-220-1-9:
----------
already_configured:
True
comment:
Already configured.
diff:
loaded_config:
result:
True
mkh@sw220-1-9> show configuration vlans vlan888
{master:0}
mkh@sw220-1-9>
Debug shows the same
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a"><lock-configuration/></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a"><lock-configuration/></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a">
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a">
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31efeac10>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e"><load-configuration format="text" action="set"><configuration-set>set vlans vlan888 vlan-id 888
</configuration-set></load-configuration></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e"><load-configuration format="text" action="set"><configuration-set>set vlans vlan888 vlan-id 888
</configuration-set></load-configuration></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f160fd0>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176"><get-configuration compare="rollback" rollback="0" format="text"/></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176"><get-configuration compare="rollback" rollback="0" format="text"/></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176">
<configuration-information>
<configuration-output>
</configuration-output>
</configuration-information>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176">
<configuration-information>
<configuration-output>
</configuration-output>
</configuration-information>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f0edb80>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[DEBUG ] Discarding the config
[DEBUG ] {'result': True, 'comment': '', 'already_configured': False, 'loaded_config': '', 'diff': ''}
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761"><load-configuration compare="rollback" rollback="0"/></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761"><load-configuration compare="rollback" rollback="0"/></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f160f70>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4"><unlock-configuration/></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4"><unlock-configuration/></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4">
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4">
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f0ed430>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO ] [host 10.220.1.9 session-id 82730] Requesting 'CloseSession'
[DEBUG ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e"><nc:close-session/></nc:rpc>
[DEBUG ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG ] [host 10.220.1.9 session-id 82730] Sending message
[INFO ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e"><nc:close-session/></nc:rpc>]]>]]>
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e">
<ok/>
</rpc-reply>
[DEBUG ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e">
<ok/>
</rpc-reply>
One more point that the same set file+ net commands are working for another Junos code different than 18, tested on 20.4 and 15.1 with no similar problem.
I added this to Salt-sproxy project as show below, amd ensured that the problem is neither with napalm nor netconf, which were tested individually and worked:
https://github.com/mirceaulinic/salt-sproxy/issues/255
Appreciate any support and best regards,
Mou
