multiple api paths support and updates (#131)

* multiple api paths support and updates

Signed-off-by: Tomas Herfert <herfik>

* sanity fix

Signed-off-by: Tomas Herfert <herfik>

* another sanity fix

Signed-off-by: Tomas Herfert <herfik>

* Apply suggestions from code review

Co-authored-by: Felix Fontein <felix@fontein.de>

Signed-off-by: Tomas Herfert <herfik>
Co-authored-by: Tomas Herfert <herfik>
Co-authored-by: Felix Fontein <felix@fontein.de>

changelogs/fragments/131-api.yml

@@ -0,0 +1,15 @@
+minor_changes:
+  - api_modify, api_info - support API paths ``ipv6 address``, ``ipv6 dhcp-server``, ``ipv6 dhcp-server option``, ``ipv6 route``, ``queue tree``, ``routing ospf area``, ``routing ospf area range``, ``routing ospf instance``, ``routing ospf interface-template``, ``routing pimsm instance``, ``routing pimsm interface-template``
+    (https://github.com/ansible-collections/community.routeros/pull/131).
+  - api_modify, api_info - support for fields ``blackhole``, ``pref-src``, ``routing-table``, ``suppress-hw-offload``, ``type``, ``vrf-interface`` in ``ip route`` path
+    (https://github.com/ansible-collections/community.routeros/pull/131).
+known_issues:
+  - api_modify - when limits for entries in ``queue tree`` are defined as human readable - for example ``25M`` -, the configuration will be correctly set in ROS, but the module will indicate the item is changed on every run even when there was no change done. This is caused by the ROS API which returns the number in bytes - for example ``25000000`` (which is inconsistent with the CLI behavior). In order to mitigate that, the limits have to be defined in bytes (those will still appear as human readable in the ROS CLI)
+    (https://github.com/ansible-collections/community.routeros/pull/131).
+  - "api_modify, api_info - ``routing ospf area``, ``routing ospf area range``, ``routing ospf instance``, ``routing ospf interface-template`` paths are not fully implemeted for ROS6 due to the significat changes between ROS6 and ROS7
+    (https://github.com/ansible-collections/community.routeros/pull/131)."
+bugfixes:
+  - "api_modify - ``queue interface`` path works now
+    (https://github.com/ansible-collections/community.routeros/pull/131)."
+  - "api_modify - ``ip route`` entry can be defined without the need of ``gateway`` field, which is correct for unreachable/blackhole type of routes
+    (https://github.com/ansible-collections/community.routeros/pull/131)."

plugins/module_utils/_api_data.py

@@ -400,16 +400,22 @@ PATHS = {
     ('ip', 'route'): APIData(
         fully_understood=True,
         fields={
+            'blackhole': KeyInfo(can_disable=True),
             'check-gateway': KeyInfo(can_disable=True),
             'comment': KeyInfo(can_disable=True, remove_value=''),
             'disabled': KeyInfo(default=False),
             'distance': KeyInfo(),
             'dst-address': KeyInfo(),
-            'gateway': KeyInfo(required=True),
+            'gateway': KeyInfo(),
+            'pref-src': KeyInfo(),
+            'routing-table': KeyInfo(default='main'),
             'route-tag': KeyInfo(can_disable=True),
             'routing-mark': KeyInfo(can_disable=True),
             'scope': KeyInfo(),
+            'suppress-hw-offload': KeyInfo(default=False),
             'target-scope': KeyInfo(),
+            'type': KeyInfo(can_disable=True, remove_value='unicast'),
+            'vrf-interface': KeyInfo(can_disable=True),
         },
     ),
     ('ip', 'route', 'vrf'): APIData(
@@ -437,45 +443,78 @@ PATHS = {
         },
     ),
     ('routing', 'ospf', 'instance'): APIData(
-        unknown_mechanism=True,
-        # primary_keys=('default', ),
+        fully_understood=True,
+        primary_keys=('name', ),
         fields={
-            'default': KeyInfo(),
-            'disabled': KeyInfo(),
-            'distribute-default': KeyInfo(),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'disabled': KeyInfo(default=False),
             'domain-id': KeyInfo(can_disable=True),
             'domain-tag': KeyInfo(can_disable=True),
-            'in-filter': KeyInfo(),
-            'metric-bgp': KeyInfo(),
-            'metric-connected': KeyInfo(),
-            'metric-default': KeyInfo(),
-            'metric-other-ospf': KeyInfo(),
-            'metric-rip': KeyInfo(),
-            'metric-static': KeyInfo(),
+            'in-filter-chain': KeyInfo(can_disable=True),
+            'mpls-te-address': KeyInfo(can_disable=True),
             'mpls-te-area': KeyInfo(can_disable=True),
-            'mpls-te-router-id': KeyInfo(can_disable=True),
             'name': KeyInfo(),
-            'out-filter': KeyInfo(),
-            'redistribute-bgp': KeyInfo(),
-            'redistribute-connected': KeyInfo(),
-            'redistribute-other-ospf': KeyInfo(),
-            'redistribute-rip': KeyInfo(),
-            'redistribute-static': KeyInfo(),
-            'router-id': KeyInfo(),
+            'originate-default': KeyInfo(can_disable=True),
+            'out-filter-chain': KeyInfo(can_disable=True),
+            'out-filter-select': KeyInfo(can_disable=True),
+            'redistribute': KeyInfo(can_disable=True),
+            'router-id': KeyInfo(default='main'),
             'routing-table': KeyInfo(can_disable=True),
             'use-dn': KeyInfo(can_disable=True),
+            'version': KeyInfo(default=2),
+            'vrf': KeyInfo(default='main'),
         },
     ),
     ('routing', 'ospf', 'area'): APIData(
-        unknown_mechanism=True,
-        # primary_keys=('default', ),
+        fully_understood=True,
+        primary_keys=('name', ),
         fields={
-            'default': KeyInfo(),
-            'area-id': KeyInfo(),
-            'disabled': KeyInfo(),
-            'instance': KeyInfo(),
+            'area-id': KeyInfo(default='0.0.0.0'),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'default-cost': KeyInfo(can_disable=True),
+            'disabled': KeyInfo(default=False),
+            'instance': KeyInfo(required=True),
             'name': KeyInfo(),
-            'type': KeyInfo(),
+            'no-summaries': KeyInfo(can_disable=True),
+            'nssa-translator': KeyInfo(can_disable=True),
+            'type': KeyInfo(default='default'),
+        },
+    ),
+    ('routing', 'ospf', 'area', 'range'): APIData(
+        fully_understood=True,
+        primary_keys=('area', 'prefix', ),
+        fields={
+            'advertise': KeyInfo(default=True),
+            'area': KeyInfo(),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'cost': KeyInfo(can_disable=True),
+            'disabled': KeyInfo(default=False),
+            'prefix': KeyInfo(),
+        },
+    ),
+    ('routing', 'ospf', 'interface-template'): APIData(
+        fully_understood=True,
+        fields={
+            'area': KeyInfo(required=True),
+            'auth': KeyInfo(can_disable=True),
+            'auth-id': KeyInfo(can_disable=True),
+            'auth-key': KeyInfo(can_disable=True),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'cost': KeyInfo(default=1),
+            'dead-interval': KeyInfo(default='40s'),
+            'disabled': KeyInfo(default=False),
+            'hello-interval': KeyInfo(default='10s'),
+            'instance-id': KeyInfo(default=0),
+            'interfaces': KeyInfo(can_disable=True),
+            'networks': KeyInfo(can_disable=True),
+            'passive': KeyInfo(can_disable=True),
+            'prefix-list': KeyInfo(can_disable=True),
+            'priority': KeyInfo(default=128),
+            'retransmit-interval': KeyInfo(default='5s'),
+            'transmit-delay': KeyInfo(default='1s'),
+            'type': KeyInfo(default='broadcast'),
+            'vlink-neighbor-id': KeyInfo(can_disable=True),
+            'vlink-transit-area': KeyInfo(can_disable=True),
         },
     ),
     ('routing', 'ospf-v3', 'instance'): APIData(
@@ -512,6 +551,40 @@ PATHS = {
             'type': KeyInfo(),
         },
     ),
+    ('routing', 'pimsm', 'instance'): APIData(
+        fully_understood=True,
+        primary_keys=('name', ),
+        fields={
+            'afi': KeyInfo(default='ipv4'),
+            'bsm-forward-back': KeyInfo(),
+            'crp-advertise-contained': KeyInfo(),
+            'disabled': KeyInfo(default=False),
+            'name': KeyInfo(),
+            'rp-hash-mask-length': KeyInfo(),
+            'rp-static-override': KeyInfo(default=False),
+            'ssm-range': KeyInfo(),
+            'switch-to-spt': KeyInfo(default=True),
+            'switch-to-spt-bytes': KeyInfo(default=0),
+            'switch-to-spt-interval': KeyInfo(),
+            'vrf': KeyInfo(default="main"),
+        },
+    ),
+    ('routing', 'pimsm', 'interface-template'): APIData(
+        fully_understood=True,
+        fields={
+            'disabled': KeyInfo(default=False),
+            'hello-delay': KeyInfo(default='5s'),
+            'hello-period': KeyInfo(default='30s'),
+            'instance': KeyInfo(required=True),
+            'interfaces': KeyInfo(can_disable=True),
+            'join-prune-period': KeyInfo(default='1m'),
+            'join-tracking-support': KeyInfo(default=True),
+            'override-interval': KeyInfo(default='2s500ms'),
+            'priority': KeyInfo(default=1),
+            'propagation-delay': KeyInfo(default='500ms'),
+            'source-addresses': KeyInfo(can_disable=True),
+        },
+    ),
     ('snmp', 'community'): APIData(
         unknown_mechanism=True,
         # primary_keys=('default', ),
@@ -824,6 +897,19 @@ PATHS = {
             'tcp-syncookies': KeyInfo(default=False),
         },
     ),
+    ('ipv6', 'address'): APIData(
+        fully_understood=True,
+        fields={
+            'address': KeyInfo(),
+            'advertise': KeyInfo(default=True),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'disabled': KeyInfo(default=False),
+            'eui-64': KeyInfo(default=False),
+            'from-pool': KeyInfo(),
+            'interface': KeyInfo(required=True),
+            'no-dad': KeyInfo(default=False),
+        },
+    ),
     ('ipv6', 'settings'): APIData(
         single_value=True,
         fully_understood=True,
@@ -1467,6 +1553,36 @@ PATHS = {
             'use-peer-dns': KeyInfo(default=True),
         },
     ),
+    ('ipv6', 'dhcp-server'): APIData(
+        fully_understood=True,
+        primary_keys=('name', ),
+        fields={
+            'address-pool': KeyInfo(required=True),
+            'allow-dual-stack-queue': KeyInfo(can_disable=True, remove_value=True),
+            'binding-script': KeyInfo(can_disable=True, remove_value=''),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'dhcp-option': KeyInfo(default=''),
+            'disabled': KeyInfo(default=False),
+            'insert-queue-before': KeyInfo(can_disable=True, remove_value='first'),
+            'interface': KeyInfo(required=True),
+            'lease-time': KeyInfo(default='3d'),
+            'name': KeyInfo(),
+            'parent-queue': KeyInfo(can_disable=True, remove_value='none'),
+            'preference': KeyInfo(default=255),
+            'rapid-commit': KeyInfo(default=True),
+            'route-distance': KeyInfo(default=1),
+            'use-radius': KeyInfo(default=False),
+        },
+    ),
+    ('ipv6', 'dhcp-server', 'option'): APIData(
+        fully_understood=True,
+        primary_keys=('name',),
+        fields={
+            'code': KeyInfo(required=True),
+            'name': KeyInfo(),
+            'value': KeyInfo(default=''),
+        },
+    ),
     ('ipv6', 'firewall', 'address-list'): APIData(
         fully_understood=True,
         primary_keys=('address', 'list', ),
@@ -1562,6 +1678,7 @@ PATHS = {
         },
     ),
     ('ipv6', 'route'): APIData(
+        fully_understood=True,
         fields={
             'bgp-as-path': KeyInfo(can_disable=True),
             'bgp-atomic-aggregate': KeyInfo(can_disable=True),
@@ -1570,14 +1687,19 @@ PATHS = {
             'bgp-med': KeyInfo(can_disable=True),
             'bgp-origin': KeyInfo(can_disable=True),
             'bgp-prepend': KeyInfo(can_disable=True),
+            'type': KeyInfo(can_disable=True, remove_value='unicast'),
+            'blackhole': KeyInfo(can_disable=True),
             'check-gateway': KeyInfo(can_disable=True),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
             'disabled': KeyInfo(),
-            'distance': KeyInfo(),
+            'distance': KeyInfo(default=1),
             'dst-address': KeyInfo(),
             'gateway': KeyInfo(),
             'route-tag': KeyInfo(can_disable=True),
-            'scope': KeyInfo(),
-            'target-scope': KeyInfo(),
+            'routing-table': KeyInfo(default='main'),
+            'scope': KeyInfo(default=30),
+            'target-scope': KeyInfo(default=10),
+            'vrf-interface': KeyInfo(can_disable=True),
         },
     ),
     ('mpls', ): APIData(
@@ -1977,11 +2099,31 @@ PATHS = {
         },
     ),
     ('queue', 'interface'): APIData(
+        primary_keys=('interface', ),
+        fully_understood=True,
+        fixed_entries=True,
+        fields={
+            'interface': KeyInfo(required=True),
+            'queue': KeyInfo(required=True),
+        },
+    ),
+    ('queue', 'tree'): APIData(
         primary_keys=('name', ),
         fully_understood=True,
         fields={
-            'name': KeyInfo(required=True),
-            'queue': KeyInfo(required=True),
+            'bucket-size': KeyInfo(default='0.1'),
+            'burst-limit': KeyInfo(default=0),
+            'burst-threshold': KeyInfo(default=0),
+            'burst-time': KeyInfo(default='0s'),
+            'comment': KeyInfo(can_disable=True, remove_value=''),
+            'disabled': KeyInfo(default=False),
+            'limit-at': KeyInfo(default=0),
+            'max-limit': KeyInfo(default=0),
+            'name': KeyInfo(),
+            'packet-mark': KeyInfo(default=''),
+            'parent': KeyInfo(required=True),
+            'priority': KeyInfo(default=8),
+            'queue': KeyInfo(default='default-small'),
         },
     ),
     ('interface', 'ethernet', 'switch'): APIData(

plugins/modules/api_info.py

@@ -112,19 +112,30 @@ options:
         - ip traffic-flow
         - ip traffic-flow ipfix
         - ip upnp
+        - ipv6 address
         - ipv6 dhcp-client
+        - ipv6 dhcp-server
+        - ipv6 dhcp-server option
         - ipv6 firewall address-list
         - ipv6 firewall filter
         - ipv6 nd prefix default
+        - ipv6 route
         - ipv6 settings
         - mpls
         - mpls ldp
         - port firmware
         - ppp aaa
         - queue interface
+        - queue tree
         - radius incoming
         - routing bgp instance
         - routing mme
+        - routing ospf area
+        - routing ospf area range
+        - routing ospf instance
+        - routing ospf interface-template
+        - routing pimsm instance
+        - routing pimsm interface-template
         - routing rip
         - routing ripng
         - snmp

plugins/modules/api_modify.py

@@ -117,19 +117,30 @@ options:
         - ip traffic-flow
         - ip traffic-flow ipfix
         - ip upnp
+        - ipv6 address
         - ipv6 dhcp-client
+        - ipv6 dhcp-server
+        - ipv6 dhcp-server option
         - ipv6 firewall address-list
         - ipv6 firewall filter
         - ipv6 nd prefix default
+        - ipv6 route
         - ipv6 settings
         - mpls
         - mpls ldp
         - port firmware
         - ppp aaa
         - queue interface
+        - queue tree
         - radius incoming
         - routing bgp instance
         - routing mme
+        - routing ospf area
+        - routing ospf area range
+        - routing ospf instance
+        - routing ospf interface-template
+        - routing pimsm instance
+        - routing pimsm interface-template
         - routing rip
         - routing ripng
         - snmp