YANG for Dummies!¶
Most people are excited to hear about the new projects from NetworkToCode, but quickly discover that there are a lot of moving parts that present a steep learning curve. Community contributions will be critical to the success and adoption of these projects. The groundwork has already been laid for IOS and JunOS drivers, so I want to document the path to onboarding completely new drivers. This is standalone work that you can replicate without making changes to upstream libraries. I'll demonstrate this based on David Barroso's great article titled "YANG for dummies". Most of the setup files for this tutorial were plagarized from ntc-rosetta. I am definitely standing on the backs of giants!
To play along with this notebook, first run:
git clone [email protected]:dgjustice/ntc-dummies.git && cd ntc-dummies
make build_test_container
make jupyter
Then open your browser to localhost:8888.
Let's start by creating some file structure... (This is already done in the container)
-
Create yang model in
data/yang/dummies/models/star-wars/napalm-star-wars.yang
-
dummies └── models ├── napalm-star-wars │ └── napalm-star-wars.yang └── napalm-star-wars-library.json
- Copied from Napalm-Automation
-
Create the library file (minus the comment):
cat napalm-star-wars-library.json
{ "ietf-yang-library:modules-state": { "module-set-id": "6bd894f2-9168-484e-a0bf-f3ed38d864f9", "module": [ { "name": "napalm-star-wars", "revision": "2019-08-31", # <- This key *must* be present, see RFC8040 "conformance-type": "implement" } ] } }
-
head napalm-star-wars.yang
// module name module napalm-star-wars { // boilerplate yang-version "1"; namespace "https://napalm-yang.readthedocs.io/yang/napalm-star-wars"; prefix "napalm-star-wars"; revision "2019-08-31" { # <- This *must* match the revision above. description "initial version"; reference "0.0.1"; } ...
-
The dummy device¶
Our dummy device is going to implement configuration through YAML files. That's absurd you say! YAML is a horrible format! I don't disagree, but it's suitable for pet examples and it lives up to the "dummy" device name. A more practical example might be to parse a text file for device configuration or interface state. If you are using something like textfsm, you will end up with a dictionary just like the following example. The end goal regardless of the input is to start our parsing process with some kind of structured data - most commonly JSON/Python dictionaries. This could be easily replicated with a dictionary returned from a TextFSM process, for example.
def read_file() -> str:
"""Read a file and return the text.
This method will be attached to our parser as a helper below."""
with open("data/star_wars/universe.yml") as f:
config = f.read()
return config
print(read_file())
The parser¶
This is the code that will parse the model. Please refer to the official Yangify docs for a complete explanation and walkthrough. Also see existing implementations at the ntc-rosetta Github page. As you are developing and debugging your own parsers, keep in mind that pdb
is your friend!
from typing import Any, Dict, Iterator, Optional, Tuple, cast
import json
from ruamel.yaml import YAML
from ntc_rosetta.helpers import json_helpers as jh
from yangify import parser
from yangify.parser.text_tree import parse_indented_config
from yangify.parser import Parser, ParserData
def to_yaml(config: str) -> Dict[str,Any]:
yaml = YAML()
config_data = yaml.load(config)
return config_data
class IndividualData(Parser):
class Yangify(ParserData):
path = "/napalm-star-wars:individual"
def extract_elements(self) -> Iterator[Tuple[str, Dict[str, Any]]]:
for person in jh.query("individuals", self.native, default=[]):
yield "individual", cast(Dict[str, Any], person)
def name(self) -> Optional[str]:
return jh.query("name", self.yy.native)
def age(self) -> int:
return jh.query("age", self.yy.native)
def affiliation(self) -> bool:
return jh.query("affiliation", self.yy.native)
class Universe(Parser):
class Yangify(ParserData):
path = "/napalm-star-wars:universe"
individual = IndividualData
class DummyParser(parser.RootParser):
"""
DummyParser expects as native data a dictionary where the `universe`
key is reserved for the device configuration.
"""
class Yangify(parser.ParserData):
def init(self) -> None:
self.root_native = to_yaml(self.root_native)
self.native = self.root_native["universe"]
universe = Universe
The driver¶
We need to register the driver with rosetta so we can make use of some its utility methods. We'll need to overload the get_datamodel
function so we can load our custom models.
YANG models¶
ntc-rosetta is a framework, so we are able to bring-our-own-yang models to the party. Simply tell yangson where to find the data, and we're off to the races (or the next step of debugging)!
from ntc_rosetta.drivers.base import Driver
from yangson.datamodel import DataModel
import pathlib
# This part looks just like the ios and junos drivers in ntc-rosetta
class DummyDriverNapalmStarWars(Driver):
parser = DummyParser
translator = None
datamodel_name = "napalm_star_wars"
# These are the overloads. Please see the file structure and notes
# in the introduction
@classmethod
def get_data_model(cls) -> DataModel:
base = pathlib.Path("/ntc_dummies/data/yang")
lib = base.joinpath("dummies/models/napalm-star-wars-library.json")
path = [
base.joinpath("dummies/models/napalm-star-wars"),
]
return DataModel.from_file(lib, path)
@classmethod
def get_datamodel(cls) -> DataModel:
if cls._datamodel is None:
cls._datamodel = cls.get_data_model()
return cls._datamodel
Check the datamodel¶
Let's take a peek at the YANG model to make sure the above worked correctly
# If everything worked
dummy_driver = DummyDriverNapalmStarWars()
# Let's see if we properly loaded the DataModel from the new YANG file
print(dummy_driver.get_datamodel().ascii_tree())
Parse it¶
Finally, we should be able to parse our "configuration" file and get a yang datamodel of our star-wars characters!
config_data = read_file()
# Alternate options
# my_universe = DummyParser(dummy_driver.get_datamodel(), native=config_data)
# print(my_universe.process().raw_value())
parsed = dummy_driver.parse(native=config_data)
print(json.dumps(parsed.raw_value(), indent=4))
We did it!¶
At this point, we have a populated data model that we can manipulate using Yangson. For a deeper dive into manipulating populated data models, please see my other tutorial at the ntc-rosetta project
Do I need to use Rosetta?¶
Strictly speaking, no. You can look at the commented lines in the previous cell for an alternative route (you'll have to separate the get_datamodel
method). Rosetta is an interface that nicely binds these dependencies together, but your particular use case may have different requirements.
Translate it¶
This about wraps up the story. We'll write a simple translator to spit out yaml back out into a "native config" format. Keep in mind that this is a pet example. In production, you will need to handle replaces and merges carefully as well as add tests.
from yangify import translator
from yangify.translator import Translator, TranslatorData
from yangify.translator.config_tree import ConfigTree
class Individual(Translator):
"""
Implements /napalm-star-wars:universe
"""
class Yangify(translator.TranslatorData):
def pre_process(self) -> None:
self.result = self.result.new_section("")
def name(self, value: Optional[str]) -> None:
self.yy.result.add_command(f" - {value}")
def age(self, value: Optional[int]) -> None:
self.yy.result.add_command(f" {value}")
def affiliation(self, value: Optional[str]) -> None:
self.yy.result.add_command(f" {value[0]}")
class Universe(Translator):
class Yangify(translator.TranslatorData):
def pre_process(self) -> None:
self.result.new_section("individuals:")
individual = Individual
class DummyTranslator(translator.RootTranslator):
class Yangify(translator.TranslatorData):
def init(self) -> None:
self.root_result = ConfigTree()
self.result = self.root_result
self.result.add_command("---")
self.result.add_command("universe:")
def post(self) -> None:
self.root_result = self.root_result.to_string()
universe = Universe
Validate the translation¶
This is a sanity check to see if we're on the right path. So far, everything looks good.
# update the driver
DummyDriverNapalmStarWars.translator = DummyTranslator
dummy_driver = DummyDriverNapalmStarWars()
# Alternate options
# my_universe = DummyTranslator(dummy_driver.get_datamodel(), candidate=parsed.raw_value())
# print(my_universe.process())
translated = dummy_driver.translate(candidate=parsed.raw_value())
print(translated)
Play with the model¶
At this point, we've built a large pet example. Does this have any practical merit? The time I've spent on this surely hopes so! Why don't we do what we've always wished for in a Star Wars movie - Yoda the mercenary!!!
irt = parsed.datamodel.parse_resource_id("/napalm-star-wars:universe/individual=Yoda")
current_data = parsed.root.goto(irt)
print("Current Yoda: ", json.dumps(current_data.raw_value(), indent=2))
modify_data = current_data.raw_value()
modify_data['affiliation'] = 'napalm-star-wars:MERCENARY'
stub = current_data.update(modify_data, raw=True)
print("Candidate Yoda: ", json.dumps(stub.raw_value(), indent=2))
stub.validate()
What broke?!¶
Nothing!!! This is by design. If you'll take a look at the beginning of the article, we only loaded the first Napalm-Star-Wars model. We didn't load the extended universe. Fine, we'll simply push him all the way to the dark side. :)
modify_data['affiliation'] = 'napalm-star-wars:EMPIRE'
stub = current_data.update(modify_data, raw=True)
print("Candidate Yoda: ", json.dumps(stub.raw_value(), indent=2))
stub.validate()
print("Validated!")
translated = dummy_driver.translate(candidate=stub.top().raw_value())
print(translated)
Thank you!¶
As I mentioned earlier, I stand on the backs of giants. Many thanks to Ken Celenza and the folks at Network To Code, David Barroso, and others involved in these community projects!!!