Author

I made this blog to help redo my CCNP, during a period of unemployment.

My study topics are mostly ENCOR, ENARSI, and ENSLD.

I’ve been in the Networking Industry since 2012, I’ve been an IT Person since 1999.

I’ve worked at Cisco on-and-off over the last 10y+, on teams like TAC for IOS-XR, or HTTS PS TAC for Route/Switch.

• Linkedin.
• Resume.

Email: ariadne@haske.org

License

This work is dedicated to the public domain via Creative Commons CC0.

How to study

I do something like:

1. Skim material (This is a vibe check).
2. Go back over material, slower, with an eye towards the exam topics.
3. Take basic notes.
4. Pick a topic, create a lab with a working example.
5. Find source materials, or more examples of a given technology.
6. Talk with others about the topic.
7. Understand the existing lab, look at debugs, logs, packets.
8. Read more source Material.
9. Take more advanced notes documenting observations about behavior.
10. Understand where this concept fits with other related concepts …

Image by Tidema - CC BY 4.0

References

Bloom’s taxonomy - Wikipedia

How To Take Notes

The Process

Consistency

If all your notes are in spiral bound notebooks or 3x5 cards, that’s OK, but pick a system and stick with it.

Detail

Cite your sources. You need enough to remember where something comes from.

Summarize

Do not re-create source materials.

• Record facts
• Summarize and reduce scope
• Fit your notes into your whole understanding (synthesis)

An hour long meeting? One page of notes. Don’t overdo it.

Myths

I Don’t Need To Take Notes. I Can Remember Everything Just Fine

Note down what you will not remember

• Bit counts
• Packet headers
• Tiny specific interactions

You can find these details via

• Reading white-papers
• Reading RFCs
• Reading forum posts
• Doing labs
• Finding outputs
• Collecting packets
• Collecting debugs

I Don’t Look At My Notes After I Take Them

Don’t take notes on stuff you’ll remember, take notes on stuff you’ll forget.

Learn what kinds of details you forget.

The notes aren’t for you six months from now, the notes are for you, six years from now.

I Can’t Take Notes Digitally, I Need To Write Them Out

You can take handwritten notes, just OCR them into digitals. The AI is decent at this.

If typing is painful consider a much nicer keyboard and something like Dvorak

References

MIT - Notes and Notebooks - Mayfield Handbook of Technical and Scientific Writing

Berkeley - Academic Skills Resource Library | Athletic Study Center

Harvard - Note-Taking – Academic Resource Center

How to Make a Blog Like This

This blog follows the Documentation as Code ethos.

• Create an account on GitHub
• Write articles in Markdown
• Learn Git Fundamentals
• Use Git
• Use GitHub Pages and Actions to Deploy mdBook
  • My Deploy script on GitHub

There is some AI use here, the articles are 98% human written.

I use Anthropic’s tools

Where LLMs are used

• OCR tasks
  • Diagram-to-Code tasks
• Reforming tables
• Lint
• Spelling and grammar
• Finding potential technical errors
• Rubber Ducking
• Vibe Coding
• Debugging
• Lab Implementation

Where LLMs are never used

• Lab Design
• Note Taking
• Prose
• Technical Writing

Preprocessors

mdBook is used to turn CommonMark into html.

These tools extend the html features, typically with JavaScript.

Mermaid

• Turns text into diagrams
• Allows version control for diagrams
• SVG Adaptive
  • Resize Nicely
  • Light and Dark theme adaptive

Mermaid is a binary js file that gets copied to the root of the repo.

curl -sL https://cdn.jsdelivr.net/npm/mermaid@11.5.0/dist/mermaid.min.js -o mermaid.min.js

Then mdbook needs to be rebuilt:

mdbook build

SVGBOB

• Sometimes
• Converts ASCII Art to a SVG
• It does this on build

gitinfo

• Injects Git Metadata into the rendered HTML articles
  • Commit
  • Date of Commit
  • Link to Commit

… Onto every webpage

Editable Extras

Is a tiny js file that modifies book.js to allow editing console examples directly on the webpage.

This lives in the root of the repo.

Git

Git is a collaborative code tool. It allows distributed development, with yourself or others.

The database that holds the code, the history of changes to the code is a repository.

Previous version control tools used diffs, which are changes or deltas between files “lines 38 to 42 have changed”, Git is snapshot based.

Every file being tracked is hashed with SHA-1, each and every time a commit is made. Git always knows if files have been changed.

The list of files and their hashes is called the Index.

Terms

Repository

• .git/
• AKA, Repo
• AKA, the object store.
• Content Addressable Filesystem.
• Key-value based. All of Git is key-value.
• De-Duplicated. If two files have the same SHA-1, git stores 1 blob.
  • Git stores: blobs, trees, commits, and tags.
• Interacted with almost purely via git.

Commits

• Takes a snapshot of the current Index
• SHA-1 entity that points to a specific tree.
• Metadata: author, commiter, timestamp, messages, parent commit.
• Commits are backwards chains.
  • All commits store the hash of previous commits.

Tree

• A directory snapshot
  • Mode (file type and permissions)
  • Name (file or directory)
  • SHA-1 (the hash)
  • Trees point to blobs and other trees.

Blob

• A key-value pair that represents a file.
• 82da472f6d00dc5f0a651f33ebb320aa9c7b08d0 LICENSE
• The SHA-1 is used to find the compressed content of LICENSE

Branch

• A named pointer to a specific commit.
• Commits are stored as nodes on a DAG.
• Merge commits have two or more parents.
• The init commit has no parents.

HEAD

• A named pointer to the current branch.
• In detached HEAD state, points directly to a commit.
• Works like the playhead on a tapedeck.

Working Tree

• AKA, project directory.
• AKA, user directory.
• AKA, your files
• This is where project files are modified.
• Directory inside git init was ran.

The Index

• .git/index
• AKA, staging
• AKA, cache.
• AKA, pre-commit.
• AKA, git’s files.
• Invoked with git add
• Adding a file does two things:
  • SHA-1 of the file, adding it to the Index.
  • Writes the blob to .git/objects/
• Files in the Index are tracked.

Local Files Overview

Git tries to avoid touching working tree files. When a file is added, a snapshot is taken at that time.

Image courtesy of Derrick Stolee.

Branch Example

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "C"
   commit id: "D"
   commit id: "HEAD → E"

A, B, C, D are previous commits.

The working tree contains the files from commit E.

I think of it like a tape deck, the HEAD can be played backwards or forwards.

commit moves the HEAD forward, and reset moves the HEAD backwards.

Remote and Local Overview

A basic example: a local repo, and a remote repo.

flowchart TD
    Remote --> Local
    Local --> Remote

The default reference for the remote repo is origin.

Local branches are referenced by name e.g. main

flowchart TD
    Remote[origin/main] --> Local
    Local[main] --> Remote

Pull

To update the local repo to match the remote, use pull

flowchart TD
    Remote[origin/main] -->|pull| Local
    Local[main] --> Remote

git pull

Push

To update the remote repo to match the local, use push

flowchart TD
    Remote[origin/main] --> Local
    Local[main] --> |push|Remote

git push

Git Operations

image courtesy of Reddit

References

Git - User-Manual Documentation

Git Merge

Warning

Do not perform pull or a merge with uncommitted changes.

merge is designed to combine committed changes. You can lose work.

How Does Merge Work

E is the last shared commit.

Merge will play back the commits on both branches. If neither set of commits touches the same parts of the file, the branches are merged, and HEAD moves to the end.

gitGraph
   commit id: "D"
   commit id: "E"
   branch feature
   checkout feature
   commit id: "A"
   commit id: "B"
   commit id: "MERGE_HEAD → C"
   checkout main
   commit id: "F"
   commit id: "HEAD → G"

After the merge

gitGraph
   commit id: "D"
   commit id: "E"
   branch feature
   checkout feature
   commit id: "A"
   commit id: "B"
   commit id: "C"
   checkout main
   commit id: "F"
   commit id: "G"
   merge feature id: "HEAD → H"

Merge Types

Fast Forward

The default merge is called fast forward, or FF.

FF can be used when there are no local changes or local commits.

The repo was cloned previously.

Before fetch.

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "HEAD → C"

After fetch.

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "HEAD → C"
   commit id: "D"
   commit id: "E"

After merge.

git merge origin/main

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "C"
   commit id: "D"
   commit id: "HEAD → E"

Pull

pull combines fetch and merge. pull assumes a fast forward merge.

Before pull.

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "HEAD → C"

After pull.

gitGraph
   commit id: "A"
   commit id: "B"
   commit id: "C"
   commit id: "D"
   commit id: "HEAD → E"

True Merge

This happens trying to update the part of a file, someone else has already updated.

1. HEAD pointer stays the same

2. MERGE_HEAD pointer on the branch to be merged.

3. What can be merged cleanly is merged.

4. Index records three versions of the files. Ancestor, HEAD and MERGE_HEAD

   • Files are merged in the working directory with conflict markers <<<<<<, =======, >>>>>>`

5. A ref called AUTO_MERGE gets created.

Conflict Resolution

After the merge, Git will say what files need to be resolved. This is called Conflict Resolution.

To resolve:

1. Open the file.
2. Find the Markers.
3. Delete lines you don’t want.
4. Delete the markers.

Two way Conflict

In Conflict.

                   Here are lines that are either unchanged from the common        
                   ancestor, or cleanly resolved because only one side changed,    
                   or cleanly resolved because both sides changed the same way.    
                   <<<<<<< yours:sample.txt                                        
Your change  ┌───  Conflict resolution is hard;                                    
             └───  let's go shopping.                                              
                   =======                                                         
    Theirs    ───  Git makes conflict resolution easy.                             
                   >>>>>>> theirs:sample.txt                                       
                   And here is another line that is cleanly resolved or unmodified.

Resolved, kept local change.

                   Here are lines that are either unchanged from the common        
                   ancestor, or cleanly resolved because only one side changed,    
                   or cleanly resolved because both sides changed the same way.    
Your change  ┌───  Conflict resolution is hard;                                    
             └───  let's go shopping.                                              
                   And here is another line that is cleanly resolved or unmodified.

Three Way Conflict

zdiff3 shows the conflict with the original text, adding the ||||||| marker.

In Conflict

                 Here are lines that are either unchanged from the common        
                 ancestor, or cleanly resolved because only one side changed,    
                 or cleanly resolved because both sides changed the same way.    
                 <<<<<<< yours:sample.txt                                        
      Yours ┌──  Conflict resolution is hard;                                    
            └──  let's go shopping.                                              
                 ||||||| base:sample.txt                                         
Original    ┌──  or cleanly resolved because both sides changed identically.     
 (Ancestor) └──  Conflict resolution is hard.                                    
                 =======                                                         
     Theirs  ──  Git makes conflict resolution easy.                             
                 >>>>>>> theirs:sample.txt                                       
                 And here is another line that is cleanly resolved or unmodified.

Resolved, kept their lines.

                 Here are lines that are either unchanged from the common        
                 ancestor, or cleanly resolved because only one side changed,    
                 or cleanly resolved because both sides changed the same way.    
     Theirs  ──  Git makes conflict resolution easy.                             
                 And here is another line that is cleanly resolved or unmodified.

Conflicts Resolved

This will also check the merge status.

git merge --continue

Abort A Merge

merge --abort

References

Git - git-merge Documentation

Git Branch

Always name branches as lowercase, as case-sensitive branches do not work across all operating systems.

Create a New Branch

git branch --copy new-branch

Create a Branch From a Specific Branch

git branch --copy new-branch old-branch

Delete a Branch

Only works if it’s fully merged.

git branch --delete dev

Delete a Remote Branch

Only works if it’s fully merged.

git branch --delete --remotes dev

Move a Branch

git branch --move old-branch new-branch

See all Branches

git branch --all

References

github - Git branch name - case sensitive or insensitive? - Stack Overflow

Git - git-branch Documentation

Git Switch

This command was added to Git in 2019 (2.23), after user feedback that checkout was destructive in some cases.

switch updates the working tree, and index to match the new branch.

Important

The destructive form of this command is --discard-changes.

Make A New Branch

git switch -c newbranch

Make A New Branch From A Specific Commit

git switch -c newbranch <commit>

References

Git - git-switch Documentation

Git Push

Git has a default setup for where it plans to perform the push. This is the push.default.

The push.default contains the refspec.

Refspec

The refspec maps a local branch to a remote branch.

This is inside of .git/config

[remote "origin"]
        url = git@github.com:ariadne-notes/network-notes.git
        fetch = +refs/heads/*:refs/remotes/origin/*

Format of Refspec

This is the fetch line from above.

• Format: +<src>:<dst>
  • + force update (non-fast-forward allowed)
  • <src> source ref
  • <dst> destination ref

Updating History

Making a change on the remote, to the commit history itself requires force.

The safer way to do this is per-branch, not the entire repo.

git push origin +main

References

Git - git-push Documentation

Git - The Refspec

Git Log

Top down, the normal view.

git log --oneline

Bottom up, good for rebasing.

git log --oneline --reverse

Decorate with active branches.

git log --oneline --decorate --graph

Decorate with all branches, even rebased ones.

git log --oneline --decorate --graph --all

Before Rebasing

This can help find a commit in the event you need it again.

git log --reverse --oneline --date=short --format="%ad | %h | %an | %s" > commit-before-rebasing.txt

References

Git - git-log Documentation

Git Rebase

Caution

rebase modifies history.

Do not rebase commits that others may have based work on.

Two branches, the start.

---
config:
  gitGraph:
    parallelCommits: true
---
gitGraph LR:
   commit id: "A"
   commit id: "B"
   commit id: "C"
   branch dev
   checkout dev
   commit id: "HEAD → E"
   checkout main
   commit id: "HEAD → D"

Normally, with two branches, we’d do a merge

Merging

---
config:
  gitGraph:
    parallelCommits: true
---
gitGraph LR:
   commit id: "A"
   commit id: "B"
   commit id: "C"
   branch dev
   checkout dev
   commit id: "HEAD → E"
   checkout main
   commit id: "D"
   merge dev id: "HEAD → F"

F is the commit the combines the diff of the endpoints of the branches main and dev.

The dev branch is just hanging out.

Rebasing

---
config:
  gitGraph:
    parallelCommits: true
---
gitGraph LR:
   commit id: "A"
   commit id: "B"
   commit id: "C"
   branch dev
   checkout dev
   commit id: "HEAD → E"
   checkout main
   commit id: "HEAD → D"

Finding the common ancestor to both branches, C we go “This is the common base, just play the diffs forward from both branches onto main”

git checkout dev
git rebase main

---
config:
  gitGraph:
    parallelCommits: true
---
gitGraph LR:
   commit id: "A"
   commit id: "B"
   commit id: "C"
   commit id: "HEAD/main → D"
   commit id: "HEAD/dev → E"

Now the HEAD for two branches main and dev are on the same branch. Main can be FF’d to move the HEAD.

git checkout main
git merge dev

---
config:
  gitGraph:
    parallelCommits: true
---
gitGraph LR:
   commit id: "A"
   commit id: "B"
   commit id: "C"
   commit id: "D"
   commit id: "HEAD/main, dev → E"

References

Git - Rebasing

Git - git-rebase Documentation

Git Reset

Reset changes what commit HEAD points to. This is one way to undo a bad local commit, or a series of bad local commits.

Reset always modifies history.

Soft Reset

AKA, Squashing

• Move HEAD.
• Do not modify the Index.
• Do not modify the Working Tree.

This is useful to undo local commits then re-play them as one commit.

e.g, you don’t need to git add the files are already in the Index.

git reset --soft HEAD~3
git commit

Mixed Reset

Creates Intermediate Commits.

• Move HEAD.
• Reset Index to HEAD
• Do not modify the Working Tree.

Maybe you made 5 local commits but you’d prefer if it was 2.

git reset --mixed HEAD~5
git add file1.c
git add file2.c
git commit -m "Intermediate Commit 1"
git add file3.c
git add file4.c
git commit -m "Final commit"

Hard Reset

Caution

This erases uncommitted work. This creates dangling local commits.

• Move HEAD
• Reset Index to HEAD
• Reset Working Tree to HEAD

git fetch origin
git reset --hard origin/main

References

Git - git-reset Documentation

Git RM

Used to remove files from the Index, in the event they are deleted from the working tree, and need to be deleted from future commits.

Example

I’ve already deleted this file from the Working Tree, now I want to remove it from the Index.

ariadne@tesseract:~/git/network-notes$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   .github/workflows/mdbook.yml
        modified:   book.toml
        deleted:    src/New Text Document.txt

git rm --cached src/New Text Document.txt

References

Git - git-rm Documentation

Git Commit

Ordinary Commit

git commit -m "Here is what I did"

Amend - Change the Last Commit Message

Warning

Do not amend commits that have already been published.

git commit --amend -m "This is what I wished it said"

All

• Automatically stages all files known to the Index
• Does not add new files

git commit --all -m "Used when you don't want to manually add files

References

Git - git-commit Documentation

Git Restore

This command was added to Git in 2019 (2.23), after user feedback that checkout was destructive in some cases.

Important

These are destructive commands.

These commands should be used with -- otherwise Git doesn’t know if you want a branch, directory, or file.

-- means file.

Restore from Index

• Copy a file to the working tree from the Index.

A file is staged.

git add hello.c

Work happens (in the working tree) now this file is broken.

git restore -- hello.c

Restore the Index

• Copy a file to the Index from the current commit..

This “unstages” a file, the version in HEAD is restored to Index.

Equivalent to undoing git add

git restore --staged -- hello.c

Restore a file to working tree from origin

git restore --source origin/main -- hello.c

References

Git - git-restore Documentation

What is git restore and how is it different from git reset? - Stack Overflow

Git Ignore

The .gitignore file is useful for things that should not be included in Git.

# - Comments

/ - Directories, the first / is the root of the repo.

Example

This ignores the file itself, and the directory /book which is the build artifacts when mdbook is run.

ariadne@tesseract:~/git/network-notes$ cat .gitignore 
.gitignore
/book

References

Git - gitignore Documentation

IPv4

Address Ranges

This Network

The 0.0.0.0 address literally means “when an app requests connectivity to 0.0.0.0, bind to every interface running IP and make it work”.

References

How Class E addresses solve for IP address exhaustion in GKE | Google Cloud Blog

IPv4 Packet Header

packet
0-3: "Version"
4-7: "IHL"
8-15: "Type Of Service"
16-31: "Total Length"
32-47: "Identification"
48-50: "Flags"
51-63: "Fragment Offset"
64-71: "Time to Live"
72-79: "Protocol"
80-95: "Header Checksum"
96-127: "Source Address"
128-159: "Destination Address"
160-183: "Options"
184-191: "Padding"

Flags

  Flags:  3 bits

    Various Control Flags.

      Bit 0: reserved, must be zero
      Bit 1: (DF) 0 = May Fragment,  1 = Don't Fragment.
      Bit 2: (MF) 0 = Last Fragment, 1 = More Fragments.

          0   1   2
        +---+---+---+
        |   | D | M |
        | 0 | F | F |
        +---+---+---+

IPv4 Address Planning

1. Define the requirements
2. Plan the ipv4 range
3. Document the plan

Example Standards

• Statically assign network infrastructure
• User devices are DHCP.
• .1 is the default gateway
• Third Octet: The vlan ID. 10.0.150.0/24 is vlan 150.
• Forth Octet: Address Assignment type. 1 to 99 are static IPs, 100-200 are DHCP.

Figure out NAT

• How many IPs are needed?
• Do any devices need to be reachable from the outside?

Best Practices

• Address internal hosts with RFC1918 addresses.

Recommended Networks

References

IPv4 Interactive Map - bgp.tools

Classful Networking

Never speak out loud “This is a class C address” unless you are in a 1990s movie.

Early Internet addressing (1980s) the IP itself indicated the subnet mask, by using the High Order bits. There were only three network sizes.

Internet Address starts with 0-127? You must have one network with 16 million hosts.

Internet Address starts with 128-191? You must have one network with 65 thousand hosts.

Internet Address starts with 192-223? You must have one network with 254 hosts.

We know now this was a terrible idea, but parts of it stay with us today.

• /24 (from class C) is a very popular prefix. This is said out loud as “A twenty four”.
• /16 (from class B) is a very popular prefix. This is said out loud as “A sixteen”.
• All v4 multicast addresses still start with 1110

If someone gives you an IP, you can never guess its network without also being told the mask.

My personal favorite is 8.8.8.8/32 Google DNS it works by Anycast!

Classful Networking, the RFC

RFC 791

Internet Protocol
Specification

  Addressing

    To provide for flexibility in assigning address to networks and
    allow for the  large number of small to intermediate sized networks
    the interpretation of the address field is coded to specify a small
    number of networks with a large number of host, a moderate number of
    networks with a moderate number of hosts, and a large number of
    networks with a small number of hosts.  In addition there is an
    escape code for extended addressing mode.

    Address Formats:

      High Order Bits   Format                           Class
      ---------------   -------------------------------  -----
            0            7 bits of net, 24 bits of host    a
            10          14 bits of net, 16 bits of host    b
            110         21 bits of net,  8 bits of host    c
            111         escape to extended addressing mode

Subnetting with Fingers

I just memorize these sequences, ungainly, but works.

Decimal masks - 128, 192, 224, 240, 248, 252, 254, 255

Wildcard masks - 127, 63, 31, 15, 7, 3, 1, 0

Subnet Sizes (going up) - 256, 512, 1024, 2048

Subnet Sizes (going down) - 128, 64, 32, 16, 8, 4, 2, 1

Subnet Examples

/24 is 256 IPs. Most gear complains if you use .0 or .256, so we say 254 usable hosts.

/30 is the 1990s way of addressing a point-to-point link, which wastes two IPs.

/31 is exactly two IPs. This is the best subnet for point-to-point links.

/32 is a single address. We call these host routes. 8.8.8.8/32 is Google’s DNS.

References

Using 31-Bit Prefixes on IPv4 Point-to-Point Links

Subnetting with the Box Method

For visual learners.

For a /24 network, thats 256 addresses, or … this box.

Start With A /24

Write the first IP in the top corner, the last IP in the bottom corner.

Cut In Half (1)

Write the last number in the bottom corner, next number in the top corner.

Two /25 networks.

Cut In Half (2)

• One /25
• Two /26 networks.

Cut In Half (3)

• One /25
• Four /27

Cut In Half (4)

• One /25
• Two /27
• Four /28

Subnetting with Binary

• A byte is eight bits 0000 0000.
• A nibble is four bits 0000.
• A bit is on or off.

Powers of 2: 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048

These are the groups.

Start with a /24, eight bits, or values from 0 to 256.

0000 0000

100 Hosts

A subnet needs 100 hosts, to represent that in binary, we’d need at least 7 bits, or 128.

A /24 would be too large, (256 IPs), remove a bit, and we get a /25.

10.0.0.0/25.

Borrowing One Bit

0000 0000

1000 0000

20 Hosts

We already spent the first portion of the address space, from 0 to 127.

0b0000 0000 is 0.

0b1000 0000 is 128.

How many bits to represent 20 hosts? at least 5, or 32.

So we start with 128, and use at least five bits.

0b1000 1000

IPv6

There are a lot of fundamentals changed with v6.

• 128 bits, more space.
• Globally Unique, less need for NAT.
• Fixed header length, v4 has variable length headers
• Optional, Option headers
• SLAAC. Stateless host addressing, with the router advertising the subnet prefix.
• Flow Labeling (for QoS)
• Routers cannot fragment packets
• Hosts can perform MTU path discovery
• Hosts can have multiple addresses and even multiple subnets
• Mobile IPv6 lets mobile nodes remain reachable
• No broadcast traffic

Header

RFCs really like groups of 32

Deering & Hinden            Standards Track                     [Page 3]

RFC 2460                   IPv6 Specification              December 1998

3.  IPv6 Header Format

┌───────┬───────────────┬───────────────────────────────────────┐
│Version│ Traffic Class │           Flow Label                  │
├───────┴───────────────┴───────┬───────────────┬───────────────┤
│         Payload Length        │  Next Header  │   Hop Limit   │
├───────────────────────────────┴───────────────┴───────────────┤
│                                                               │
│                                                               │
│                                                               │
│                         Source Address                        │
│                                                               │
│                                                               │
│                                                               │
├───────────────────────────────────────────────────────────────┼
│                                                               │
│                                                               │
│                                                               │
│                      Destination Address                      │
│                                                               │
│                                                               │
│                                                               │
└───────────────────────────────────────────────────────────────┘

Bits

Addresses are 128 bits.

A standard IPv6 address takes this form.

FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

The groups are called hextets, as they are made with hex characters.

(I used F because IPv6 is hexadecimal)

• Each 0xFFFF is 16 bits.
• Each 0xFF is 8 bites, or a byte.
• Each 0xF is 4 bits, or a nibble.

Alternative Representation of a IPv6 Address.

RFC 4291 allows this:

0:0:0:0:0:0:10.0.0.1

::10.0.0.1

These are only API addresses to represent to a IPv6 app, that this is an IPv4 host.

They don’t go anywhere.

IPv6 Address Block

All globally routeable IPv6 addresses fit into this block.

2000::/3

IPv6 Special Address Blocks

IPv6 Documentation Prefixes

References

RFC 2460 - Internet Protocol, Version 6 (IPv6)

IPv6 Subnetting

Not really used by individual sites, but if you get a block from a RIR, this is how to subnet it.

In the US ARIN handles requests for v6 space.

Could be useful to work with NPTv6

Blessed Subnets

The easy v6 networks to subnet fall on hex digit boundaries of /4.

For these, see this chart.

/40

You’re given 3fff::/20, make some /40 networks.

A /40 is two hextets, and two hex digits worth of bits.

• 3fff:0:0000::/40
• 3fff:0:0100::/40
• 3fff:0:0200::/40
• 3fff:0:0300::/40
• 3fff:0:0400::/40
• 3fff:0:0500::/40

which becomes

• 3fff:0::/40
• 3fff:0:100::/40
• 3fff:0:200::/40
• 3fff:0:300::/40
• 3fff:0:400::/40
• 3fff:0:500::/40

/44

You’re given 3fff::/20, make some /44 networks.

A /44 is two hextets, and three hex digits worth of bits.

• 3fff:0:0000::/44
• 3fff:0:0010::/44
• 3fff:0:0020::/44
• 3fff:0:0030::/44
• 3fff:0:0040::/44
• 3fff:0:0050::/44

which becomes

• 3fff:0::/44
• 3fff:0:10::/44
• 3fff:0:20::/44
• 3fff:0:30::/44
• 3fff:0:40::/44
• 3fff:0:50::/44

/48

You’re given 3fff::/20, make some /48 networks.

A /48 is three hextets, worth of bits.

• 3fff:0:0::/48
• 3fff:0:1::/48
• 3fff:0:2::/48
• 3fff:0:3::/48
• 3fff:0:4::/48

Cursed Subnets

What if instead, we try and subnet, inside a hex digit?

First, we’d have to know what every hex digit is in binary.

Then we’d want to figure out where the boundaries are for bits borrowed:

/49

You’re given 3fff::/20, make some /49 networks.

A /49 is three hextets, plus one binary bit.

• 3fff:0:0:0::/49
• 3fff:0:0:8000::/49
• 3fff:0:1:0::/49
• 3fff:0:1:8000::/49
• 3fff:0:2:0000::/49

/50

You’re given 3fff::/20, make some /50 networks.

A /50 is three hextets, plus two binary bits.

• 3fff:0:0:0::/50
• 3fff:0:0:4000::/50
• 3fff:0:0:8000::/50
• 3fff:0:0:C000::/50
• 3fff:0:1:0000::/50

/51

You’re given 3fff::/20, make some /51 networks.

A /51 is three hextets, plus three binary bits.

• 3fff:0:0:0::/51
• 3fff:0:0:2000::/51
• 3fff:0:0:4000::/51
• 3fff:0:0:6000::/51
• 3fff:0:0:8000::/51

References

IPv6 address - Wikipedia

IPv6 Hextext Boundaries

For any of these, to subnet them, just change the final hex digit.

Groups of 64

/64   FFFF:FFFF:FFFF:FFFF::
/128  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Groups of 32

/32   FFFF:FFFF::
/64   FFFF:FFFF:FFFF:FFFF::
/96   FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/128  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

As groups of 16

/16   FFFF::
/32   FFFF:FFFF::
/48   FFFF:FFFF:FFFF::
/64   FFFF:FFFF:FFFF:FFFF::
/80   FFFF:FFFF:FFFF:FFFF:FFFF::
/96   FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/112  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/128  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Groups of 8

/8    FF::
/16   FFFF::
/24   FFFF:FF::
/32   FFFF:FFFF::
/40   FFFF:FFFF:FF::
/48   FFFF:FFFF:FFFF::
/56   FFFF:FFFF:FFFF:FF::
/64   FFFF:FFFF:FFFF:FFFF::
/72   FFFF:FFFF:FFFF:FFFF:FF::
/80   FFFF:FFFF:FFFF:FFFF:FFFF::
/88   FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/96   FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/104  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/112  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/120  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/128  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Groups of 4

/4    F::
/8    FF::
/12   FFF::
/16   FFFF::
/20   FFFF:F::
/24   FFFF:FF::
/28   FFFF:FFF::
/32   FFFF:FFFF::
/36   FFFF:FFFF:F::
/40   FFFF:FFFF:FF::
/44   FFFF:FFFF:FFF::
/48   FFFF:FFFF:FFFF::
/52   FFFF:FFFF:FFFF:F::
/56   FFFF:FFFF:FFFF:FF::
/60   FFFF:FFFF:FFFF:FFF::
/64   FFFF:FFFF:FFFF:FFFF::
/68   FFFF:FFFF:FFFF:FFFF:F::
/72   FFFF:FFFF:FFFF:FFFF:FF::
/76   FFFF:FFFF:FFFF:FFFF:FFF::
/80   FFFF:FFFF:FFFF:FFFF:FFFF::
/84   FFFF:FFFF:FFFF:FFFF:FFFF:F::
/88   FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/92   FFFF:FFFF:FFFF:FFFF:FFFF:FFF::
/96   FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/100  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:F::
/104  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/108  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFF::
/112  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF::
/116  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:F::
/120  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FF::
/124  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFF::
/128  FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

IPv6 Address Architecture

Common subnets found in v6 BGP global table.

Found in the APNIC BGP in 2025 pdf.

If a /24 is the minimum accepted route prefix size in IPv4, what is the comparable size in IPv6?

There appears to be no common consensus position here. The default action many for network operators appears to have no minimum size filter at all. In theory, that would imply that a /128 route object would be accepted across the entire IPv6 DFZ. A more pragmatic observation is that a /32 would be assuredly accepted by all networks, and it appears that many network operators believe that a /48 is also generally accepted. Given that a /48 is the most common prefix size in today’s IPv6 network, this view appears to be widespread. We also see prefixes smaller in size than a /48 in the routing table with /49, /52, /56 and /64 prefixes present in the IPv6 BGP routing table. 0.7% all advertised prefixes are more specific than a /48.

Site Planning

Since each site gets it’s own /48, there are 65k networks to work with.

References

BGP in 2025 | APNIC Blog

IPv6 Neighbor Discovery

RA messages are sent with ICMPv6. These can carry options, so they’ve been extended to carry RDNSS information.

The router can say “Here is the DNS info”.

Terms

RA — Router Advertisement

In v6, routers can just advertise the prefix of the attached subnet and options like RDNSS.

RDNSS — Recursive DNS Server

Router Advertisements

From RFC 4861: Neighbor Discovery for IP version 6 (IPv6).

packet
  0-7: "Type"
  8-15: "Code"
  16-31: "Checksum"
  32-39: "Cur Hop Limit"
  40: "M"
  41: "O"
  42-47: "Reserved"
  48-63: "Router Lifetime"
  64-95: "Reachable Time"
  96-127: "Retrans Timer"
  128-159: "Options ..."

RDNS Server Option

From RFC 8106: IPv6 Router Advertisement Options for DNS Configuration

packet
  0-7: "Type"
  8-15: "Length"
  16-31: "Reserved"
  32-63: "Lifetime"
  64-127: "Addresses of IPv6 Recursive DNS Servers"

Packet Capture

Taken from my home router 13-July-2026, I’ve modified the IPs.

Frame 46: Packet, 174 bytes on wire (1392 bits), 174 bytes captured (1392 bits) on interface <removed>
Ethernet II, Src: Routerboardc_ef:69:14 (48:a9:8a:ef:69:14), Dst: IPv6mcast_01 (33:33:00:00:00:01)
Internet Protocol Version 6, Src: fe80::4aa9:8aff:feef:6914, Dst: ff02::1
Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    Code: 0
    Checksum: 0xa34c [correct]
    [Checksum Status: Good]
    Cur hop limit: 0
    Flags: 0x00, Prf (Default Router Preference): Medium
    Router lifetime (s): 1800
    Reachable time (ms): 0
    Retrans timer (ms): 0
    ICMPv6 Option (Source link-layer address : 48:a9:8a:ef:69:14)
    ICMPv6 Option (Recursive DNS Server 2001:db8:12:15::6 2001:db8:12:15::7)
        Type: Recursive DNS Server (25)
        Length: 5 (40 bytes)
        Reserved
        Lifetime: 1800 (30 minutes)
        Recursive DNS Servers: 2001:db8:12:15::6
        Recursive DNS Servers: 2001:db8:12:15::7
    ICMPv6 Option (Recursive DNS Server fe80::4aa9:8aff:feef:6914)
        Type: Recursive DNS Server (25)
        Length: 3 (24 bytes)
        Reserved
        Lifetime: RDNSS address MUST no longer be used (0) (0 seconds)
        Recursive DNS Servers: fe80::4aa9:8aff:feef:6914
    ICMPv6 Option (Prefix information : 2001:db8:12:15::/64)
        Type: Prefix information (3)
        Length: 4 (32 bytes)
        Prefix Length: 64
        Flag: 0xc0, On-link Flag (L), Autonomous Address Configuration Flag (A)
            1... .... = On-link Flag (L): Set
            .1.. .... = Autonomous Address Configuration Flag (A): Set
            ..0. .... = Router Address Flag (R): Not set
            ...0 .... = DHCPv6-PD Preferred Flag (P): Not set
            .... 0000 = Reserved: 0
        Valid Lifetime: 2592000 (30 days)
        Preferred Lifetime: 604800 (7 days)
        Reserved
        Prefix: 2001:db8:12:15::

Wireshark

BPF:ip6 protochain 58

References

RFC 4861: Neighbor Discovery for IP version 6 (IPv6) | RFC Editor

RFC 8106: IPv6 Router Advertisement Options for DNS Configuration | RFC Editor

IPv4 to IPV6 Migration Strategies

• Dual-stack: AKA, Native Mode. Both at the same time. Twice the administrative overhead
• Tunneling: Send the IPv6 packets to tunnel to cross a IPv4 core.
• Translation: Make the IPv6 packets back into IPv4 packets.
• Hybrid Model: Uses ISATAP
• ISATAP: Intra-Site Automatic Tunneling Addressing
• Service Blocks: Combing tunnels with dual-stack.

Native Mode

Both stacks are running on clients. The clients request A and AAAA records near simultaneously.

Whichever DNS records return first decides if the next set of flows are v4 or v6.

This strategy is called “Happy Eyeballs” after the RFC. Do not punish users by “preferring” one protocol or another.

Do whatever is fastest.

Tunneling

• Reduces MTU by 20 bytes.
• The preferred tunnel is GRE.

Automatic Tunnels

6to4

• Uses 2002::/16

6RD

• The ISP uses its own addressing.

ISATAP — Intra-site Automatic Tunnel Addressing Protocol

• Uses these 32 bits to denotes ISATAP 0000:5EFE

Translation

• Much like NAT

DNS64 And NAT64

DNS64

• DNS creates synthetic AAAA records, so a v4 client can connect to a v6 service.

NAT64

• A router translates IPv6 traffic to IPv4 traffic.

References

Happy Eyeballs Version 2: Better Connectivity Using Concurrency

Connection of IPv6 Domains via IPv4 Clouds

Framework for IPv4/IPv6 Translation

ISATAP

• Packet-in-packet technology
• Mostly historical
• Used within one site, not an Internet Technology
• “v6 islands within a v4 ocean”
• Treats the v4 network as NBMA
• ISATAP devices must be dual stacked
• Adds 20-bytes of overhead
• Works as long as v4 routing works
• Stateless

Theory

Use v4 connectivity to emulate a NBMA network. Wrap the v6 packet inside v4.

The routers perform ISATAP

         ┌─ v6 only ─┐         ┌────────────  v4 only ───────────────┐         ┌─ v6 only ─┐
         ▼           ▼         ▼                                     ▼         ▼           ▼
┌────────┐           ┌─────────┐            ┌─────────┐              ┌─────────┐           ┌────────┐
│client-1│           │    R1   │            │   R2    │              │    R3   │           │client-2│
│   v6   ├───────────┤  v4/v6  ├────────────┤    v4   ├──────────────│  v4/v6  ├───────────┤   v6   │
└────────┘           └─────────┘            └─────────┘              └─────────┘           └────────┘
                             ISATAP                              ISATAP

                              Interface                           Interface

                           ┌───────────┬───────────┐
                           │ v6-packet │ v4-header │ ───────►
                           └───────────┴───────────┘

The clients perform ISATAP

           ┌──────────────────────────────────  v4 only  ───────────────────────────────────┐
           ▼                                                                                ▼
  ┌────────┐           ┌─────────┐            ┌─────────┐              ┌─────────┐          ┌────────┐
  │client-1│           │   R1    │            │   R2    │              │   R3    │          │client-2│
  │ v4/v6  ├───────────┤    v4   ├────────────┤    v4   ├──────────────┤    v4   │──────────┤  v4/v6 │
  └────────┘           └─────────┘            └─────────┘              └─────────┘          └────────┘
        ISATAP                                                                        ISATAP

         Interface                                                                     Interface

        ┌───────────┬───────────┐
        │ v6-packet │ v4-header │ ────────►
        └───────────┴───────────┘

ISATAP Interface in v6

All ISATAP interfaces look like this. The giveaway is 0000:5EFE in the host portion.

If a router sees a v6 destination that looks like this, it’s an ISATAP packet.

                    64 bits                              32 bits                   32 bits

┌───────────────────────────────────────────────┬───────────────────────────┬──────────────────────┐
│  Global Unicast Prefix                        │        0000:5EFE          │  IPv4 of ISATAP Link │
└───────────────────────────────────────────────┴───────────────────────────┴──────────────────────┘

Terms

PRL — Potential Router List

• Routers that could perform ISATAP

ISATAP Interface

• Dual-stack
  • Has the v6 IP

ISATAP v4 Address

• NBMA address
• How to reach this device via ISATAP

ISATAP v6 Address

• End-to-end v6 address
• Must embed the v4 address

References

ISATAP - Wikipedia

RFC 5214: Intra-Site Automatic Tunnel Addressing Protocol (ISATAP)

ISATAP.org - Wayback Machine

IP Routing Configuration Guide - ISATAP Tunnel Support for IPv6 Support - Cisco

6RD

6to4

• The base standard
• Requires a specific prefix

6RD

• The current standard
• Allows any global prefix

RG — Router Gateway

• The CPE the ISP provides to connect to their network

BNG — Broadband Network Gateway

• This is the Customer aggregation node
• Aggregate thousands of connections

NMS — Network Management System

• Provides a place to see and configure the network

AAA — Authentication, Authorization, Accounting

• This is how the ISP controls who can access the network
• Controls QoS Deployments

BR

• Terminates the 6RD tunnels
• AKA, Border Relay

What does 6RD Solve?

“We can’t deploy v6, because our BNG does (put-feature-here) and we need that!

This way, the customer gets dual stack without the ISP needing to upgrade a legacy deployment.

The Four Things Required To Setup

• IPv4 Common Prefix: - The high order bits every CE has in common for their v4 deployments.

If every customer was given an IP on the 10.0.0.0/24 network, the bits they would have in common is /24.

• 6RD Prefix: The v6 bits for the 6rd domain.
• 6RD PrefixLen: The length of the prefix for the 6rd domain.
• 6RD BR v4 Address: The Access Network v4 address for the BR.

(there is a neat way to derive the 6rd prefix, refer to Cisco’s IOS-XE manual, that’s closest to how it’s done.)

Sample Config (from Cisco’s PDF)

!
! This BR is reachable via the loopback.
!
interface Loopback0
ip address 10.1.1.1 255.255.255.0
!
interface Tunnel0
 tunnel source Loopback0
 tunnel mode ipv6ip 6rd
 tunnel 6rd ipv4 prefix-len 8
 tunnel 6rd prefix 2011:1001:100:/40
 ipv6 address 2011:1001:101:101::/128 anycast
!
ipv6 route 2011:1001:100::/40 Tunnel0
ipv6 route 2011:1001:101:101::/64 Null0
!
! Simple and Easy to setup!
!
#show tunnel 6rd
Interface Tunnel0:
 Tunnel Source: 10.1.1.1
 6RD: Operational, V6 Prefix: 2011:1001:100:/40
 V4 Prefix, Length: 8, Value: 10.0.0.0
 V4 Suffix, Length: 0, Value: 0.0.0.0
 General Prefix: 2011:1001:1

References

RFC 3056: Connection of IPv6 Domains via IPv4 Clouds | RFC Editor

RFC 5969: IPv6 Rapid Deployment on IPv4 Infrastructures (6rd) – Protocol Specification | RFC Editor

APNIC Conference | 6RD Enabling IPv6 Customers on an IPv4-only Network

Interface and Hardware Component Configuration Guide, Cisco IOS XE Fuji 16.9.x - IPv6 Rapid Deployment Cisco ASR 1000 Series Aggregation Services Routers - Cisco

NAT64

• Requires a DNS64 Server to translate an A record to a synthetic AAAA record.
• Requires a Upstream DNS to respond with an A record, to the DNS64 server.
• Requires a “Stateful Prefix”

sequenceDiagram
    participant PC as PC<br/>IPV6-20
    participant DNS64 as Server<br/>DNS-64
    participant DNS4 as Server<br/>DNS-4
    participant R1 as Router<br/>R1
    participant Client as Client<br/>IPV4-20

    PC->>DNS64: Request AAAA
    DNS64->>DNS4: Request A
    DNS4-->>DNS64: Answer with A
    DNS64-->>PC: Translate to Synthetic AAAA
    PC->>R1: Send with v6 prefix
    R1->>Client: Translate to v4
    Client-->>R1: Respond with v4
    R1-->>PC: Check Translations<br/>Reply with v6

Config

ipv6 unicast-routing
!
! Define NAT64 Prefix
!
nat64 prefix stateful 2001:DB8:0:64::/96
!
! Create NAT64 pool
!
nat64 v4 pool DHCP_POOL64 10.0.0.100 10.0.0.200
!
! Create ACL for NAT64 Clients
!
ipv6 access-list ACL_NAT64_CLIENTS
  permit ipv6 2001:db8:1::/48 any
!
! Match v6 clients to the v4 pool
!
nat64 v6v4 list ACL_NAT64_CLIENTS pool DHCP_POOL64 overload
!
! Enable on v6 network
!
interface GigabitEthernet3
 ipv6 address 2001:db8:1::1/64
 nat64 enable
!
! Enable on v4 network
!
interface GigabitEthernet1
 ip address 10.0.0.1 255.255.255.0
 nat64 enable

DNS64

apk update apk add unbound apk add bind-tools rc-update add unbound default

DNS64:~# cat /etc/unbound/unbound.conf 
server:
    interface: 0.0.0.0
    interface: ::0
    access-control: 0.0.0.0/0 allow
    access-control: ::/0 allow
    do-ip4: yes
    do-ip6: yes
    do-udp: yes
    do-tcp: yes
    module-config: "dns64 iterator"
    # Do not use the Well-Known-Prefix, it breaks with private v4 addresses.
    # See https://datatracker.ietf.org/doc/html/rfc6052#section-3.4
    dns64-prefix: 2001:db8:0:64::/96

forward-zone:
        name: "."
        # A different upstream server, that returns only an A record.
        forward-addr:  2001:db8:1::6

DNS4

Installing Apps

apk update
apk add unbound
apk add bind-tools
rc-update add unbound default

DNS4:~$ cat /etc/unbound/unbound.conf
server:
    interface: 0.0.0.0
    interface: ::0
    access-control: 0.0.0.0/0 allow
    access-control: ::/0 allow
    do-ip4: yes
    do-ip6: yes
    do-udp: yes
    do-tcp: yes
    local-data: "64lab.example.com. A 10.0.0.20"

Verification

R1#show nat64 translations 

Proto  Original IPv4         Translated IPv4
       Translated IPv6       Original IPv6 
----------------------------------------------------------------------------

icmp   10.0.0.20:2           [2001:db8:0:64::a00:14]:2                       
       10.0.0.100:2          [2001:db8:1:0:5054:ff:fe11:d8ec]:2              

Total number of translations: 1

The WKP (Well Known Prefix) will not work toward RFC1918 addresses. You must configure a different /96.

References

RFC 6052: IPv6 Addressing of IPv4/IPv6 Translators | RFC Editor

IP Addressing Configuration Guide, Cisco IOS XE 17.x - Stateful Network Address Translation 64 Cisco IOS XE 17 - Cisco

RFC 4291: IP Version 6 Addressing Architecture | RFC Editor

RFC 2464: Transmission of IPv6 Packets over Ethernet Networks

802.1Q

A 32 bit tag added to a Ethernet frame to multiplex VLANs

                                   ┌────── Priority Code Point(PCP)
                                   │         Used for LAN CoS
                                   │
                                   │   ┌── Drop Elgible Indicator (DEI)
                                   │   │
                                   ▼   ▼
┌───────────────────────────────┬─────┬─┬───────────────────────┐
│    Tag Protocol Identifier    │     │ │                       │
│     (TPID) Set to 0x8100      │ PCP │ │       VLAN ID         │
│                               │     │ │                       │
│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│1 2 3│4│5 6 7 8 1 2 3 4 5 6 7 8│
└───────────────────────────────┴─────┴─┴───────────────────────┘
            16 bits                3   1        12 bits

VLAN Access Control Lists

Can be IPs and/or MACs.

These work on routed or switched traffic.

Routed Flow

block
columns 1
    id1(("Ingress\nPort"))
    block:one
      id2("Port\nACL")
      id3("VLAN\nACL")
      id4("Routed\nACL")
    end
    id5(["SVI"])
    id6("Routing\nOperation")
    id7(["SVI"])
    block:two
      id8("Routed\nACL")
      id9("VLAN\nACL")
      id10("Port\nACL")
    end
    id11(("Egress\nPort"))
    id1 space:2 id2
    id1  --- id2
    id2  --- id3
    id3  --- id4
    id4  --- id5
    id5  --- id6
    id6  --- id7
    id7  --- id8
    id8  --- id9
    id9  --- id10
    id10 --- id11
    style id6 fill:#1A1,stroke:#333,stroke-width:2px

Switched Flow

The VLAN ACL is only processed once, on switching operation.

block
columns 1
    id1(("Ingress\nPort"))
    id2("Port\nACL")
    block:switching
        id3("VLAN\nACL")
        id6("Switching\nOperation")
    end
    id10("Port\nACL")
    id11(("Egress\nPort"))
    id1  --- id2
    id2  --- id3
    id3  --- id6
    id6  --- id10
    id10 --- id11
    style id6 fill:#1A1,stroke:#333,stroke-width:2px
    style switching fill:#1A1,stroke:#333,stroke-width:2px

Config

Copied from the TAC notes.

ip access-list extended TEST
 10 permit ip host 10.1.1.1 any
 20 permit ip any host 10.1.1.1
!
ip access-list extended ELSE
 10 permit ip any any
!
vlan access-map VACL 10
 match ip address TEST
 action forward
vlan access-map VACL 20
 match ip address ELSE
 action drop
!
vlan filter VACL vlan-list 10

References

Cisco Catalyst IR8340 Rugged Series Router Software Configuration Guide, Cisco IOS XE Release 17.15.x - VLAN Access Control Lists Cisco Catalyst IR8300 Rugged Series Router - Cisco

Validate Security ACLs on Catalyst 9000 Switches - Cisco

Solved: PACL and VACL processing order - Cisco Community

QoS

Terms

FIFO — First in, First out

• Default behavior

Differentiated Services

• Giving packet flows different levels of network service
• Based on classification
• AKA, DiffServ

Integrated Services

• Packet flows explicitly reserve bandwidth along a path
• Uses admission control
• AKA, IntServ

RSVP — Resource Reservation Protocol

• An IntServ Implementation

Marking

• Changing the DSCP bits in the IP header field

DSCP — Differentiated Services Code Point

• The markings of an IP packet that allows DiffServ

PHB — Per Hop Behavior

• What a node should or shouldn’t do with marked traffic.

Queuing

• Hold a packet in memory
• Delays transmission
• Expensive, because memory is expensive

LLQ — Low latency queuing

• Describes queue behavior for the EF PHB:
  • Never drop
  • Never delay
  • Send immediately
  • Police aggressively

EF — Expedited Forwarding

• The highest PHB for network data

Control Traffic

• Routing Traffic
  • OSPF, EIGRP, IS-IS, BGP, etc

CAR — Committed Access Rate

• Contractual data rate a traffic source will flow at
• SLA oriented

SLA — Service Level Agreement

• A Business agreements about data servicing requirements

WFQ — Weighted Fair Queuing

• Legacy default on serial interfaces at E1 speeds and below ~2.048 Mbps
• Sorts traffic into high bw and low bw classes.

CBWFQ — Class Based Weighted Fair Queuing

• AKA, Modular QoS
• Multiple queues
• Bandwidth limits
• Different kinds of queues, like LLQ

MQC

• Modular QoS CLI

PQ — Priority Queue

• A queue that is served first, even if other queues have been waiting longer
• An LLQ is an example of a PQ

Type of Service

How these 8 bits get used has changed over the years.

                   0 1 2 3 4 5 6 7
                  ┌─────┬─────┬─┬─┐
   RFC 791 (1981) │IP Pr│ ToS │0│0│
                  └─────┴─────┴─┴─┘

                   0 1 2 3 4 5 6 7
                  ┌─────┬───────┬─┐
  RFC 1349 (1992) │IP Pr│  TOS  │0│
                  └─────┴───────┴─┘

                   0 1 2 3 4 5 6 7
                  ┌───────────┬─┬─┐
  RFC 2474 (1998) │    DSCP   │0│0│
                  └───────────┴─┴─┘

                   0 1 2 3 4 5 6 7
                  ┌───────────┬───┐
  RFC 3168 (2001) │    DSCP   │ECN│
                  └───────────┴───┘

PHB - Per Hop Behaviors

Assured Forwarding

Assured Forwarding PHB Group

AF uses the first 6 bits to create 4 traffic classes, 4 is best.

Within those classes, there is a drop precedence, or … at what point of queue congestion should this traffic be dropped.

Used for RED, or WRED.

Four AF classes, each should get it’s own resources.

Drop

 Precedence      Class 1        Class 2        Class 3       Class 4

            ┌───────────────┬───────────────┬───────────────┬──────────────┐
   Low    │ │ AF11  001 010 │ AF21  010 010 │ AF31  011 010 │ AF41 100 010 │
   Medium │ │ AF12  001 100 │ AF22  010 100 │ AF32  011 100 │ AF42 100 100 │
   High   ▼ │ AF13  001 110 │ AF23  010 110 │ AF33  011 110 │ AF43 100 110 │
            └───────────────┴───────────────┴───────────────┴──────────────┘
             ────────────►  Importance  to Business/Net work ───────────►

Again, with DSCP

Drop

 Precedence      Class 1        Class 2        Class 3       Class 4

            ┌───────────────┬───────────────┬───────────────┬──────────────┐
   Low    │ │ AF11  DSCP 10 │ AF21  DSCP 18 │ AF31  DSCP 26 │ AF41 DSCP 34 │
   Medium │ │ AF12  DSCP 12 │ AF22  DSCP 20 │ AF32  DSCP 28 │ AF42 DSCP 36 │
   High   ▼ │ AF13  DSCP 14 │ AF23  DSCP 22 │ AF33  DSCP 30 │ AF43 DSCP 38 │
            └───────────────┴───────────────┴───────────────┴──────────────┘
             ────────────►  Importance  to Business/Network ───────────►

Yields the following formula.

DSCP = 8 (class) + 2 (drop)

QoS Consequences

LAN QoS with voice (buffer management)

• One voice packet, no voice, but modem will retrain
• Two voice packets, audio clip, fax call disconnection.
• VoIP QoS cannot be fixed by adding bandwidth. You simply cannot drop these
• packets.

QoS Commands

RFC 4594 — DiffServ Service Classes

Source: RFC 4594 (Aug 2006), updated by RFC 5865 and RFC 8622. AF drop precedence: x1=low, x2=medium, x3=high drop probability.

References

An Architecture for Differentiated Services

Cisco IOS QoS Commands

See All Matched QoS

show mls qos interface statistics

See the DSCP to CoS Map

show mls qos map cos-dscp

See QoS on a single Interface

show mls qos interface Fa0/1

Trust All QoS Markings

mls qos trust dscp

Trust QoS if it comes from a Cisco Phone

mls qos trust device cisco-phone

Spanning Trees

Algorhyme

I think that I shall never see
a graph more lovely than a tree.

A tree whose crucial property
is loop-free connectivity.

A tree which must be sure to span
so packets can reach every LAN.

First the root must be selected.
By ID it is elected.

Least cost paths from root are traced.
In the tree these paths are placed.

A mesh is made by folks like me,
then bridges find a spanning tree.

– Radia Perlman

L2 networks were getting larger, and a common mistake was creating a loop, where frame would go around infinitely.

Spanning Tree Protocol STP disables ports to break L2 loops.

STP

Terms

STP — Spanning Tree Protocol

• Frequently cited as 802.1D

Bridge

• A device that participates in the spanning tree algorithm

Root Bridge

• The bridge that wins the STP election

Bridge ID

• Three fields, next to each other
• Bridge Priority, Extension ID (the VLAN), MAC Address

BPDU — Bridge Protocol Data Unit

• The frame used in 802.1D STP

802.1D

• An IEEE standard. The oldest Ethernet STP

Root ID

• The bridge that has won and is winning the elections

Designated ports

• Sends BPDUs downstream
• AKA DP

Root Port

• Receives BPDUs, from upstream switch.
• Each bridge can have only one RP. RP is picked by port-selection-algo
• AKA, RP
• AKA, Upstream

TCN — Topology change notification

• This is its own message
• Sent by the bridge that sees a STP change, upstream via it’s RP
• One of the only upstream messages

TCA Bit — Topology Change Acknowledge

• Sent by the upstream bridge
• Lets the downstream reporting bridge know the TCN was relayed upstream
• Inside a config BPDU

TC Bit — Topology Change

• The root bridge sets the TC Bit
• Downstream bridges shorten their MAC aging timer to Forward Delay (default 15 seconds)

How STP makes a loop free topology

STP elects root and designated ports, aka RP, and DPs. It also moves STP ports into Blocking.

• A bridge can only have one RP.
• All ports on the root are DPs.
• Ports on the root bridge never enter blocking.
• Blocked ports must keep receiving BPDUs to stay blocked (the election must continue, forever)
• if two would-be DPs send and receive BPDUs.
  • There is a loop.
  • The port that has the inferior BPDU will block.

1. All bridges turn on send BPDUs on all STP ports, themselves as root.
2. STP ports (bridges) compare BPDUs.
3. Bridge with lowest Bridge ID is root, (Lowest priority, if priority is default, lowest mac, usually the oldest switch)
4. All ports on root bridge are DP, and BPDU cost field is set to zero.
5. Root sends BPDUs.
6. DPs send configuration BDPUs.
7. RPs receive configuration BPDUs.
8. Root bridge sends BPDU, cost is 0, with port identifiers set.
9. A non-root bridge can only have one RP.
10. Non-root bridge gets BPDUs. It uses the port selection Algo to pick one RP.
11. Non-root bridge starts STP elections on all other ports, by sending BPDUs. It takes the cost inside the received BPDU, and adds it’s port cost.
12. If a DP gets a BDPU, STP blocks the port if the received BPDU is better.

Port selection algo

• All choices are made based on the received BPDU.
• Modifications are made on the upstream switch.

1. Lowest cost to root.
2. Lowest system priority of advertising switch.
3. Lowest MAC of advertising switch.
4. Port Identifier Byte of advertising switch (port priority + port number)

Timers

• Hello Time is usually 2 seconds between BPDUs.

• Forward Delay is typically 15 seconds. It’s between off -> listening -> learning.

Device priority

4 bits, goes in geometric sequence starting from 0 to 61440.

switch(config)# spanning-tree vlan 60 priority ?
% Bridge Priority must be in increments of 4096.
% Allowed values are: 
  0     4096  8192  12288 16384 20480 24576 28672
  32768 36864 40960 45056 49152 53248 57344 61440

Root bridges election in spanning tree

Two bridges send each other BPDUs, they compare bridge IDs to see who will keep sending BPDUs

The bridge with the lower ID (priority + mac address) wins. The non-root-bridge copies this bridge ID into it’s BPDU, and sends that downstream.

The default for priority is 32768 or 0x80 on the wire. Because the 802.1D committee exists, the priority is this, plus the vlan ID.

Always configure a root bridge, or the oldest device with probably the lowest mac address wins the root bridge election.

Path cost

The root bridge BPDU gets stuff tack’d onto it. The root bridge advertises itself as 0 cost.

Cost is the value of the link, towards the root bridge.

 ┌───────┐                                                                   
 │  SW1  │                                                                   
 └───┬───┘                                                                   
     │                                                                       
     │                                                                       
     │  Cost in BPDU from SW1 is 0                                           
     │                                                                       
Eth0 │ ◄──── Interface is Assigned a cost of 100 by SW2 based on link Speed
 ┌───┴───┐                                                                   
 │  SW2  │                                                                   
 └───┬───┘                                                                   
Eth1 │                                                                       
     │                                                                       
     │   Cost in BPDU on-the-wire is now 100, SW2 Eth0 Cost                  
     │                                                                       
Eth0 │                                                                       
 ┌───┴───┐                                                                   
 │  SW2  │                                                                   
 └───────┘                                                                   

Portfast

For end Hosts

• Does not protect against BPDUs

Loop Prevention

Best practice is to set the root to 0 and the secondary to 4096.

STP Loop Guard

A unidirectional failure on a root or alternate port will cause spanning tree to loop, as other switches will unblock ports, and the unidirectional failure will still forward frames. To prevent this, turn on stp loop guard so … if a port doesn’t get a BPDU, it enters STP loop-inconsistent disabling the port.

This is done per interface, and is pretty tedious.

switch(config)# interface Ethernet 1/1
switch(config-if)# spanning-tree guard loop

More details here.

Port Types

Designated ports

• Send BPDUs downstream

Root Ports

• The selected port towards the root bridge
  • Lowest total cost
    • OR lowest advertised priority
      • OR Lowest advertised port ID (interface number)

Root Path Cost

Root Path Cost

What the interfaces costs + the advertised cost to the root. The root sends a cost of 0.

STP Path Calculations

spanning-tree pathcost method long

802.1D - spanning tree

The 802.1D committee wanted two learning states¹, one with and one without learning station addresses. This is why it’s more complicated.

┌─────────────┐                                                     
│     off     │                                                     
└──────┬──────┘                                                     
       │                                                            
       │  Turn on interface                                         
       ▼                                                            
┌─────────────┐                                                     
│  Listening  │ Receive + Send BPDUs                                
└──────┬──────┘                                                     
       │                                                            
       │  forward delay (default 15s)                               
       ▼                                                            
┌─────────────┐                                                     
│  Learning   │ Receive + Send BPDUs + Program CAM                  
└──────┬──────┘                                                     
       │                                                            
       │  forward delay (default 15s)                               
       ▼                                                            
┌─────────────┐                                                     
│  Forwarding │ Receive + Send BPDUs + Program CAM + Forward Frames 
└─────────────┘                                                     

BPDU frame format

Wireshark

Spanning Tree Protocol
    Protocol Identifier: Spanning Tree Protocol (0x0000)
    Protocol Version Identifier: Spanning Tree (0)
    BPDU Type: Configuration (0x00)
    BPDU flags: 0x00
        0... .... = Topology Change Acknowledgment: No
        .... ...0 = Topology Change: No
    Root Identifier: 0 / 1 / 52:54:00:c4:f3:e7
        Root Bridge Priority: 0
        Root Bridge System ID Extension: 1
        Root Bridge System ID: 52:54:00:c4:f3:e7 (52:54:00:c4:f3:e7)
    Root Path Cost: 0
    Bridge Identifier: 0 / 1 / 52:54:00:c4:f3:e7
        Bridge Priority: 0
        Bridge System ID Extension: 1
        Bridge System ID: 52:54:00:c4:f3:e7 (52:54:00:c4:f3:e7)
    Port identifier: 0x8001 < ------------------------- first byte is "port priority", the next byte is "Port Number".
    Message Age: 0
    Max Age: 20
    Hello Time: 2
    Forward Delay: 15

• Cost This is a STP BPDU.
• Two flags
  • TC
  • TC-Ack

Two bridges

• Root bridge

• Bridge that sent the BDPU

• Max Age

• Forward Hello Time

• Forward Delay

• Port Identifier

This is what the BPDU looks like on-the-wire

┌───────────────────────────────┬───────────────┬───────────────┐
│                               │               │               │
│          Protocol ID          │    Version    │   BPDU Type   │
│                               │               │               │
│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8│
└───────────────────────────────┴───────────────┴───────────────┘
             2 bytes                  1 byte         1 byte

┌───────────────┬───────────────────────────────────────────────►
│               │
│     Flag      │                    Root ID
│               │
│1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
└───────────────┴───────────────────────────────────────────────►
    1 byte                            8 bytes

◄───────────────────────────────────────────────────────────────►

                           Root ID

 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
◄───────────────────────────────────────────────────────────────►
                          8 bytes

◄───────────────┬───────────────────────────────────────────────►
                │
    Root ID     │              Root Path Cost
                │
 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
◄───────────────┴───────────────────────────────────────────────►
    8 bytes                       4 bytes

◄───────────────┬───────────────────────────────────────────────►
 Root Path Cost │
                │                Bridge ID
                │
 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
◄───────────────┴───────────────────────────────────────────────►
  4 bytes                         8 bytes

◄───────────────────────────────────────────────────────────────►

                           Bridge ID

 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
◄───────────────────────────────────────────────────────────────►
                          8 bytes

◄───────────────┬───────────────────────────────┬───────────────►
                │                               │ Message age
   Bridge ID    │           Port ID             │  (in 1/256s of a second)
                │                               │
 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8
◄───────────────┴───────────────────────────────┴───────────────►
    8 bytes                2 Bytes                   2 Bytes

◄───────────────┬───────────────────────────────┬───────────────►
                │           Max Age             │ Hello Time
   Message Age  │        (in 1/256ths)          │  (in 1/256ths of a second)
                │                               │
 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8
◄───────────────┴───────────────────────────────┴───────────────►
    2 Bytes                 2 Bytes                   2 Bytes

◄───────────────┬───────────────────────────────┬───────────────┐
                │  Forward Delay                │   Version 1   │
   Hello Time   │    (in 1/256ths of a second)  │    Length     │
                │                               │               │
 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│1 2 3 4 5 6 7 8│
◄───────────────┴───────────────────────────────┴───────────────┘
    2 Bytes                 2 Bytes                   1 Byte

┌───────────────────────────────┐
│                               │
│      Version 3 Length         │
│                               │
│1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8│
└───────────────────────────────┘
           2 Bytes

Port elections

Bridge Priority, Vlan, Bridge MAC, Port Priority, Port Number

Default settings

Who is the root?

Both bridges temporarily send BPDUs with themselves both set as root.

┌────────┐                                                                                       ┌────────┐
│        │                                                                                       │        │
│      1 │── 32768 / 1 / 52:54:00:4b:99:08 / 8001 ─────── 32768 / 1 / 52:54:00:e8:3a:ff / 8001 ──│ 1      │
│ SW1  2 │── 32768 / 1 / 52:54:00:4b:99:08 / 8002 ─────── 32768 / 1 / 52:54:00:e8:3a:ff / 8002 ──│ 2  SW2 │
│      3 │── 32768 / 1 / 52:54:00:4b:99:08 / 8003 ─────── 32768 / 1 / 52:54:00:e8:3a:ff / 8003 ──│ 3      │
│        │                                                                                       │        │
└────────┘                                                                                       └────────┘

SW1 wins with 4b. SW1 has the lower MAC address.

32768 / 1 / 52:54:00:4b:99:08 / 8001 < 32768 / 1 / 52:54:00:e8:3a:ff

Setting Bridge priority to zero

Who is the root?

Both bridges temporarily send BPDUs with themselves both set as root.

┌────────┐                                                                                       ┌────────┐
│        │                                                                                       │        │
│      1 │── 32768 / 1 / 52:54:00:4b:99:08 / 8001 ----------- 0 / 1 / 52:54:00:e8:3a:ff / 8001 ──│ 1      │
│ SW1  2 │── 32768 / 1 / 52:54:00:4b:99:08 / 8002 ----------- 0 / 1 / 52:54:00:e8:3a:ff / 8002 ──│ 2  SW2 │
│      3 │── 32768 / 1 / 52:54:00:4b:99:08 / 8003 ----------- 0 / 1 / 52:54:00:e8:3a:ff / 8003 ──│ 3      │
│        │                                                                                       │        │
└────────┘                                                                                       └────────┘

SW2 wins with 0. SW2 has the lower bridge priority.

32768 / 1 / 52:54:00:4b:99:08 / 8001 > 0 / 1 / 52:54:00:e8:3a:ff

!
! SW2
!
spanning-tree vlan 1 priority 0

Port blocking, port default

Which ports block?

┌───────────┐                                                                                       ┌───────────────┐
│           │                                                                                       │               │
│      DP 1 │── 32768 / 1 / 52:54:00:4b:99:08 / 8001 ───────────────────────────────────────────────│ 1 RP          │
│  SW1 DP 2 │── 32768 / 1 / 52:54:00:4b:99:08 / 8002 ───────────────────────────────────────────────│ 2 BLK  SW2    │
│      DP 3 │── 32768 / 1 / 52:54:00:4b:99:08 / 8003 ───────────────────────────────────────────────│ 3 BLK         │
│           │                                                                                       │               │
└───────────┘                                                                                       └───────────────┘

• All ports on root bridge are DP.
• SW2 gets three BPDUs, the best BPDU is on port 1, it has the lowest port number.
• SW2 sets the other two ports to BLK.

Port blocking, port priority

Which ports block?

┌───────────┐                                                                                       ┌───────────────┐
│           │                                                                                       │               │
│      DP 1 │── 32768 / 1 / 52:54:00:4b:99:08 / 8001 ───────────────────────────────────────────────│ 1 BLK         │
│  SW1 DP 2 │── 32768 / 1 / 52:54:00:4b:99:08 / 8002 ───────────────────────────────────────────────│ 2 BLK  SW2    │
│      DP 3 │── 32768 / 1 / 52:54:00:4b:99:08 / 0003 ───────────────────────────────────────────────│ 3 RP          │
│           │                                                                                       │               │
└───────────┘                                                                                       └───────────────┘

• All ports on root bridge are DP.
• SW2 gets three BPDUs, the best BPDU is on port 3, it has the lowest priority. 00
• SW2 sets the other two ports to BLK.

!
! SW1
!
interface 3
 spanning-tree vlan 1 port-priority 0

Port blocking, cost?

Which ports block?

┌───────────┐                                                                                       ┌───────────────┐ 
│           │                                                                                       │               │ 
│      DP 1 │── 32768 / 1 / 52:54:00:4b:99:08 / 8001 ──────────────────────────────────────Cost─4───│ 1 BLK         │ 
│  SW1 DP 2 │── 32768 / 1 / 52:54:00:4b:99:08 / 8002 ──────────────────────────────────────Cost─1───│ 2 RP   SW2    │ 
│      DP 3 │── 32768 / 1 / 52:54:00:4b:99:08 / 8003 ──────────────────────────────────────Cost─4───│ 3 BLK         │ 
│           │                                                                                       │               │ 
└───────────┘                                                                                       └───────────────┘ 

• All ports on root bridge are DP.
• SW2 gets three BPDUs, the best BPDU is on port 2, The local switch marked the received cost on that port as 1
• SW2 sets the other two ports to BLK.

!
! SW2
!
interface 2
 spanning-tree vlan 1 cost 1

Topology change notifications (TCNs)

• A TCN is a kind of BPDU message.
• There is no root ID or bridge ID.
• The TCN is sent out the RP.

Spanning Tree Protocol
    Protocol Identifier: Spanning Tree Protocol (0x0000)
    Protocol Version Identifier: Spanning Tree (0)
    BPDU Type: Topology Change Notification (0x80)

1. Bridge sees change in STP topology, sends TCN to upstream bridge.
2. Upstream sees TCN, sends a regular BDPU back with TCA bit set.
3. Upstream bridge sends TCN upstream, this continues until TCN reaches the root.
4. Root Bridge sees the TCN, marks BPDUs with TC bit set.
5. All bridges see TC, and shorten their MAC aging timer to Forward Delay (default 15 seconds).
6. Root bridge stops sending TCs.

The default for Cisco is keeping a mac-address in CAM for 300 seconds (5 minutes)

Receiving a TCN shortens the aging timer to forward delay usually 15 seconds. This means any server that is not actively sending, will have it’s traffic flooded onto that VLAN.

switch# show mac address-table aging-time 
Global Aging Time:  300

Finding TCNs

switch# show spanning-tree vlan 20 detail | s Spanning
 VLAN0020 is executing the rstp compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, sysid 20, address aabb.cc00.0100
  Configured hello time 2, max age 20, forward delay 15, transmit hold-count 6
  Current root has priority 8212, address aabb.cc00.0200
  Root port is 7 (Ethernet1/2), cost of root path is 200
  Topology change flag not set, detected flag not set
  Number of topology changes 8 last change occurred 01:07:20 ago   < ----
          from Ethernet1/2                                         < ----
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15 
  Timers: hello 0, topology change 0, notification 0, aging 300

On the device

switch# show spanning-tree vlan 20 detail | i VLAN|transitions 
 VLAN0020 is executing the rstp compatible Spanning Tree protocol
 Port 2 (Ethernet0/1) of VLAN0020 is designated forwarding 
   Number of transitions to forwarding state: 2
 Port 4 (Ethernet0/3) of VLAN0020 is alternate blocking 
   Number of transitions to forwarding state: 1
 Port 7 (Ethernet1/2) of VLAN0020 is root forwarding 
   Number of transitions to forwarding state: 2
 Port 8 (Ethernet1/3) of VLAN0020 is alternate blocking 
   Number of transitions to forwarding state: 0
 Port 12 (Ethernet2/3) of VLAN0020 is designated forwarding 
   Number of transitions to forwarding state: 2

In the logs

switch# show logging | i %LINK
*Jul  8 04:22:24.660: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 04:22:24.702: %LINK-3-UPDOWN: Interface Ethernet0/1, changed state to up
*Jul  8 04:22:24.715: %LINK-3-UPDOWN: Interface Ethernet0/2, changed state to up
*Jul  8 04:22:24.740: %LINK-3-UPDOWN: Interface Ethernet0/3, changed state to up
*Jul  8 04:22:24.769: %LINK-3-UPDOWN: Interface Ethernet1/0, changed state to up
*Jul  8 04:22:24.794: %LINK-3-UPDOWN: Interface Ethernet1/1, changed state to up
*Jul  8 04:22:24.819: %LINK-3-UPDOWN: Interface Ethernet1/2, changed state to up
*Jul  8 04:22:24.858: %LINK-3-UPDOWN: Interface Ethernet1/3, changed state to up
*Jul  8 04:22:24.888: %LINK-3-UPDOWN: Interface Ethernet2/0, changed state to up
*Jul  8 04:22:24.903: %LINK-3-UPDOWN: Interface Ethernet2/1, changed state to up
*Jul  8 04:22:24.927: %LINK-3-UPDOWN: Interface Ethernet2/2, changed state to up
*Jul  8 04:22:24.942: %LINK-3-UPDOWN: Interface Ethernet2/3, changed state to up
*Jul  8 04:22:24.965: %LINK-3-UPDOWN: Interface Ethernet3/0, changed state to up
*Jul  8 04:22:24.989: %LINK-3-UPDOWN: Interface Ethernet3/1, changed state to up
*Jul  8 04:22:25.013: %LINK-3-UPDOWN: Interface Ethernet3/2, changed state to up
*Jul  8 04:22:25.033: %LINK-3-UPDOWN: Interface Ethernet3/3, changed state to up
*Jul  8 04:22:26.685: %LINK-5-CHANGED: Interface Vlan1, changed state to administratively down
*Jul  8 04:24:58.575: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 04:25:06.138: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 04:26:59.260: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 04:27:11.982: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 04:28:43.205: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 04:31:09.988: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 04:33:53.881: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 04:34:02.140: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 05:00:52.111: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 05:00:59.749: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 05:03:48.728: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 05:03:54.050: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 05:07:04.113: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 05:07:06.713: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 05:07:31.603: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 05:07:36.280: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up
*Jul  8 05:11:32.247: %LINK-3-UPDOWN: Interface Vlan10, changed state to up
*Jul  8 06:35:29.308: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down
*Jul  8 06:35:43.756: %LINK-3-UPDOWN: Interface Ethernet0/0, changed state to up

Captures

STP-stable-state.pcap

STP-TCN-topology-change-notification.pcap

References

R. Perlman, Interconnections: Bridges, Routers, Switches, and Internetworking Protocols, 2nd ed. Boston, MA: Addison-Wesley, 1999.

Understand and Tune Spanning Tree Protocol Timers - Cisco

Layer 2 Configuration Guide, Cisco IOS-XE 17.16.X

1. Interconnections - Radia Perlman, page 67. ↩

RSTP

Discarding

• On
• Blocking data traffic
• Might be receiving BPDUs
  • STP states: Disabled, Blocking and Listening

Learning

• On
• Building the CAM table
• Only forwarding BPDUs

Forwarding

• On
• Forwarding BPDUs and Data traffic

Root port

• AKA RP
• A switch has one RP

Designated port

• AKA DP
• Each link can have only one DP

Backup port

• Typically connected to a hub
• Used on shared segments
• Provides link redundancy

Edge ports

• Edge of the STP topology, meant for hosts
• Edge ports have portfast turned on

Non-Edge ports

• Have received a BPDUs
  • Could be attached to switches

Point-to-point ports

• Connect to other RSTP switches with full duplex

Propose bit

• Added in 802.1w
• Switches compare BPDUs
  • Switch with Superior BPDU
    • Sets the Propose and DP bits

Agree Bit

• Sent in response to a Propose Bit
  • “Yeah, I agree”
  • Transition immediately to forwarding

Packet Headers

Spanning Tree Protocol
    Protocol Identifier: Spanning Tree Protocol (0x0000)
    Protocol Version Identifier: Rapid Spanning Tree (2)
    BPDU Type: Rapid/Multiple Spanning Tree (0x02)
    BPDU flags: 0x0e, Port Role: Designated, Proposal
        0... .... = Topology Change Acknowledgment: No
        .0.. .... = Agreement: No
        ..0. .... = Forwarding: No
        ...0 .... = Learning: No
        .... 11.. = Port Role: Designated (3)
        .... ..1. = Proposal: Yes
        .... ...0 = Topology Change: No
    Root Identifier: 0 / 1 / 52:54:00:eb:eb:96
    Root Path Cost: 4
    Bridge Identifier: 32768 / 1 / 52:54:00:a9:d4:07
    Port identifier: 0x8001
    Message Age: 1
    Max Age: 20
    Hello Time: 2
    Forward Delay: 15
    Version 1 Length: 0

Proposal

Like 802.1D, we have BPDUs. To speed things up some logic has been added.

Is this port full-duplex? If so, it’s point-to-point, and if so, there is a RP and DP.

No BPDUs being received? Wait for the forward delay, transition port to forwarding.

Fastest Scenario, The Would-Be DP Is Already Transmitting

Captures

RSTP-initial-bringup-fastest.pcap

References

Cisco - Spanning Tree Protocol

MST

Cisco switches provide three kinds of spanning tree modes.

switch(config)# spanning-tree mode ?
  mst         Multiple spanning tree mode
  pvst        Per-Vlan spanning tree mode
  rapid-pvst  Per-Vlan rapid spanning tree mode

The Industry has three kinds of interop.

Industry liked what Cisco was doing with “per vlan” so MST merges that feature into 802.1s.

Terms

CST — Common Spanning Tree

• For interoperability we fall back to 802.1D, with one STP

CST Root

• The one root bridge for the entire CST

MST — Multiple Spanning Trees

MSTI — MST Instance

• A group of vlans on a common MSTI

MST Region

• A group of switches with the same high-level config

MST Region Boundary

• Where a MST region sends and receives BPDUs with a different switching instance (could be STP, RSTP, MST but a different region)

MST Region Root

• MST can propagate multiple MSTIs. Each MSTI can have it’s own root

IST — Internal Spanning Tree

• Instance 0, the first instance

IST Root

• The IST root is the CST root

CIST — Common and Internal Spanning Tree

• MST can derive what STP would do, for interop
• If a switch in a MST region is connected to a much older switch, it will present it with a CST

MST Region Boundary

• Any port that connects to a 802.1D or 802.1W device

PVST Simulation

• If a MST switch is root for the whole switch topology
  • Map the IST (instance 0) onto the CST, by sending BPDUs for all the VLANS it sees on the neighbor

PVST Simulation Check

• If a MST device receives a superior BPDU
  • Shut down the port

Packet

From wireshark

Spanning Tree Protocol                                                                                  
    Protocol Identifier: Spanning Tree Protocol (0x0000)                                                
    Protocol Version Identifier: Multiple Spanning Tree (3)                                             
    BPDU Type: Rapid/Multiple Spanning Tree (0x02)                         ───────┐                     
    BPDU flags: 0x7c, Agreement, Forwarding, Learning, Port Role: Designated      │                     
    Root Identifier: 0 / 0 / 52:54:00:5f:ff:79                                    │ Inter-op Data       
    Root Path Cost: 20000                                                         │   The CST MST       
    Bridge Identifier: 4096 / 0 / 52:54:00:82:c0:7f                               │     presents outside
    Port identifier: 0x800e                                                       │       its region    
    Message Age: 1                                                                │                     
    Max Age: 20                                                                   │                     
    Hello Time: 2                                                                 │                     
    Forward Delay: 15                                                      ───────┘                     
    Version 1 Length: 0                                                                                 
    Version 3 Length: 80                                                                                
    MST Extension                                                                                       
        MST Config ID format selector: 0                       ───────┐                                 
        MST Config name: green                                        │   What MST                      
        MST Config revision: 3                                        │    shows                        
        MST Config digest: 059b580e0d7ab80bcf83df54c634d006           │      other                      
        CIST Internal Root Path Cost: 20000                           │        devices                  
        CIST Bridge Identifier: 32768 / 0 / 52:54:00:04:67:92         │          in                     
        CIST Remaining hops: 19                                       │           its                   
        MSTID 1, Regional Root Identifier 0 / 52:54:00:82:c0:7f       │            Region               
                                                               ───────┘                                 

MST keeps track of a few things:

• Root Identifier, the interop bridge for the whole topology

• Bridge Identifier, the interop field that makes a MST region appear as one bridge outside of it

• CIST Bridge, the bridge that originated the BPDU. Not visible outside of MST

• Regional Root, the bridge that is the root for green. Not visible outside of MST

Basic Config

spanning-tree mode mst

More Involved Config

default spanning-tree mst configuration
spanning-tree mst configuration
 name blue
 revision 3
 instance 1 vlan 10, 20, 30, 40

Config Validation

S21# show spanning-tree mst configuration 
Name      [red]
Revision  3     Instances configured 2

Instance  Vlans mapped
--------  ---------------------------------------------------------------------
0         1-19,21-39,41-4094
1         20,40
-------------------------------------------------------------------------------

Outputs

• A switch ff79 in a different MST region is the root for the CST.
• Our regional root c07f is one hop away.

S32# show spanning-tree mst 

##### MST0    vlans mapped:   1-9,11-19,21-29,31-39,41-4094
Bridge        address 5254.0004.6792  priority      32768 (32768 sysid 0)
Root          address 5254.005f.ff79  priority      0     (0 sysid 0)
              port    Gi3/2           path cost     20000    
Regional Root address 5254.0082.c07f  priority      4096  (4096 sysid 0)
                                      internal cost 20000     rem hops 19
Operational   hello time 2 , forward delay 15, max age 20, txholdcount 6 
Configured    hello time 2 , forward delay 15, max age 20, max hops    20

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi3/1            Desg FWD 20000     128.14   P2p 
Gi3/2            Root FWD 20000     128.15   P2p 

##### MST1    vlans mapped:   10,20,30,40
Bridge        address 5254.0004.6792  priority      32769 (32768 sysid 1)
Root          address 5254.0082.c07f  priority      1     (0 sysid 1)
              port    Gi3/2           cost          20000     rem hops 19

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi3/1            Desg FWD 20000     128.14   P2p 
Gi3/2            Root FWD 20000     128.15   P2p 

Captures

MST-region-frame.pcap

References

Cisco - Understand the Multiple Spanning Tree Protocol (802.1s)

Layer 23 - MSTP Protocol Explained: Multiple Spanning Tree in Depth

Defend Spanning Tree

STP on its own is kind of brittle, so there are lots of features to enable to make it more resilient.

DTP

DTP — Dynamic Trunking Protocol.

DTP is a Cisco proprietary point-to-point protocol, for full-duplex switchlinks.

An older feature intended to automate parts of network setup, you could set one switch to dynamic desireable and it will form trunks automatically.

The best practice is to disable this feature on trunks links with switchport nonegotiate

DTP is normally every 30 seconds.

Modes

switchport mode dynamic auto

• Send DTP
• Usually Default
• Become a trunk if the neighbor is a trunk
• Become a trunk if the neighbor is set to desireable

switchport mode access

• Probably doesn’t¹ send DTP frames
• If it does, it asks the neighbor to become an access port

switchport mode trunk

• Send DTP
• Asks the neighbor to become a trunk port

switchport mode dynamic desirable

• Become a trunk only if the neighbor can be convinced to become a trunk
• Works with trunk, desirable or auto

switchport nonegotiate

• Disable DTP
• Only works with access or trunk

Commands

show dtp
debug dtp packets

Verification

Performed on a C3560CX, running 15.2(7r)E.

No DTP with switchport mode access

switch# show dtp interface tenGigabitEthernet 1/0/7 | i Enabled
  Enabled:                                  no

References

Dynamic Trunking Protocol - Wikipedia

VLAN Configuration Guide, Cisco IOS XE 17.15.x (Catalyst 9500 Switches) - Configuring VLAN Trunks Cisco Catalyst 9500 Series Switches - Cisco

Solved: Switchport Mode Access question - Cisco Community

1. The IOS-XE guide says “negotiates” but in lab, I don’t see DTP frames on these ports. I checked on IOSv, IOL, and a C3560CX running 15.2(7r)E.
   Consensus online is this turns off DTP. ↩

Portfast

The biggest reason to configure portfast, is portfast tells spanning-tree, this isn’t a p2p port, so please don’t send a TCN. TCNs have all devices in the STP topoolgy refresh their mac-address tables.

Immediately forward traffic

802.1D waits normally 30 seconds (2x the forward delay) before forwarding traffic. This means a modern computer, with Ethernet, will be powered on, without network, waiting those 30 seconds for DHCP to complete.

PXE (booting from network) ports should always be portfast.

Enable on all access ports

spanning-tree portfast default

Enable on trunks

Some trunk ports connect to servers. Portfast can be enabled on those too.

spanning-tree portfast trunk

BPDU Guard

• Only works if the attached device sends a BPDU. Cannot prevent a switch from being attached to a port. 802.1x helps with that.

Detects A BPDU, And Err-Disables A Port

The global command only affects ports that have portfast already turned on, i.e. this is an edge feature.

switch(config)# spanning-tree portfast bpduguard default

… should be set so access ports go errdisable when a rogue switch is connected and require an operator to correct.

Seeing err-disabled Status

switch# show int status

Port      Name               Status       Vlan       Duplex  Speed Type 
[output omitted]
Et2/3                        err-disabled 1            auto   auto unknown
Et3/0                        connected    trunk        auto   auto unknown
Et3/1                        connected    1            auto   auto unknown

Turning On Automated Recovery

switch(config)# errdisable recovery cause bpduguard

Verify

switch# show errdisable recovery 
ErrDisable Reason            Timer Status
-----------------            --------------
arp-inspection               Disabled
bpduguard                    Enabled

[output omitted]
          
Interface       Errdisable reason       Time left(sec)
---------       -----------------       --------------
unicast-flood                Disabled
vmps                         Disabled
psp                          Disabled
dual-active-recovery         Disabled
evc-lite input mapping fa    Disabled
Recovery command: "clear     Disabled

Timer interval: 300 seconds

Interfaces that will be enabled at the next timeout:

Interface       Errdisable reason       Time left(sec)
---------       -----------------       --------------
Et2/3                  bpduguard          296

BPDU Filter

Maybe you don’t want a port to send BPDUs. This effectively turns off spanning-tree.

Warning

This feature melts networks. It disables spanning tree.

Caution

This is not deterministic. It works differently per port, vs globally.

From the IOS-XE config guide

The BPDU filtering feature can be globally enabled on the switch or can be enabled per interface, but the feature operates with some differences.

Enabling BPDU filtering on PortFast enabled interfaces at the global level keeps those interfaces that are in a PortFast operational state from sending or receiving BPDUs.

The interfaces still send a few BPDUs at link-up before the switch begins to filter outbound BPDUs. You should globally enable BPDU filtering on a switch so that hosts that are connected to these interfaces do not receive BPDUs. If a BPDU is received on a PortFast enabled interface, the interface loses its PortFast operational status, and BPDU filtering is disabled.

Enabling BPDU filtering on an interface without also enabling the PortFast feature keeps the interface from sending or receiving BPDUs.

Globally

If a port, is running portfast, this feature will transmit a few BPDUs when the port first turns on.

spanning-tree portfast bpdufilter default

Per Interface

This port will never send BPDUs.

interface 1
  spanning-tree bpdufilter enable

References

Layer 2 Configuration Guide, Cisco IOS XE 26.x.x (Catalyst 9300 Switches) - Configuring Optional Spanning-Tree Features Cisco IOS XE 26 - Cisco

Root Guard

Rootguard is an alternative to BPDU guard, when the port needs to participate in BPDUs, but should never receive a superior BPDU.

Normally SW1 is the root bridge

┌───────────┐                                                                                      ┌───────────────┐
│  SW1   DP ├──  0 / 1 / 52:54:00:e8:3a:ff ─────────────────────── 4096 / 1 / 52:54:00:4b:99:08  ──┤ RP     SW2    │
└───────────┘                                                                                      └───────────────┘

Someone configures SW2 to be the root by making the switch priority 0.

┌───────────┐                                                                                      ┌───────────────┐
│  SW1   RP ├──  0 / 1 / 52:54:00:e8:3a:ff ───────────────────────    0 / 1 / 52:54:00:4b:99:08  ──┤ DP     SW2    │
└───────────┘                                                                                      └───────────────┘

This can be prevented with this config. Root guard goes onto DPs.

!
! SW1
!
interface 1
  spanning tree guard root

Verification

sw1# show spanning-tree 

VLAN0001
  Spanning tree enabled protocol ieee
  Root ID    Priority    4097
             Address     5254.004f.110e
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    4097   (priority 4096 sys-id-ext 1)
             Address     5254.004f.110e
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/0               Desg BKN*4         128.1    P2p *ROOT_Inc 
Gi0/2               Desg BKN*4         128.3    P2p *ROOT_Inc 

Logs

*May  3 20:14:45.169: %SPANTREE-2-ROOTGUARD_BLOCK: Root guard blocking port GigabitEthernet0/0 on VLAN0001.  

Loop Guard

A unidirectional failure on a root or alternate port will cause spanning tree to loop.

Loopguard enforces a simple rule.

If a port was receiving BPBUs and suddenly it stops, don’t change the STP.

• This is one of the unidirectional preventatives
• This is only for switch-to-switch ports

Terms

Unidirectional Link

A failure where one side of a fiber pair is broken.

A unidirectional failure will always result in one side not getting information.

• SW1 sends BPDUs
• SW2 never gets BPDUs

Topology

• SW1 is a root bridge
• SW2 is experiencing a UL failure
• SW2 will transition Port 1 to a RP

┌────────────────────┐                            ┌─────────────────────┐
│     ┌─────────────┐│                            │ ┌────────────┐      │
│     │ Port ┌────┐ ││ BPDU ────►                 │ │┌────┐ Port │      │
│     │  1   │ TX ├─││─────────────── Fiber Cut ──│─│┤ RX │  1   │      │
│ SW1 │      └────┘ ││                            │ │└────┘      │ SW2  │
│     │  RP  ┌────┐ ││                            │ │┌────┐  DP  │      │
│     │      │ RX ├─││────────────────────────────│─│┤ TX │      │      │
│     │      └────┘ ││                ◄───── BPDU │ │└────┘      │      │
│     └─────────────┘│                            │ └────────────┘      │
└────────────────────┘                            └─────────────────────┘

Config

Default

spanning-tree loopguard default

Per Port

interface 1
  spanning-tree guard loop

References

Understand STP Loop Guard and UDLD Features - Cisco

UplinkFast

An older feature, doesn’t work with rapid-pvst, or MST.

• Bypasses listening and learning states.
• Blasts fake traffic out the UplinkFast Port to program the upstream CAM tables.
• Only recommended on the access layer.

Config

spanning-tree uplinkfast 

Validation

Port costs are raised by 3000.

access# show spanning-tree 

VLAN0001
  Spanning tree enabled protocol ieee
  Root ID    Priority    24577
             Address     5254.00bf.ea62
             Cost        3004
             Port        1 (GigabitEthernet0/0)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    49153  (priority 49152 sys-id-ext 1)
             Address     5254.007a.bd39
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  15  sec
  Uplinkfast enabled

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/0               Root FWD 3004      128.1    P2p 
Gi0/1               Altn BLK 3004      128.2    P2p 

!
! After shutting down the port in CML
!
*Jun  4 02:41:34.557: %SPANTREE_FAST-7-PORT_FWD_UPLINK: VLAN0001 GigabitEthernet0/1 moved to Forwarding (UplinkFast).

References

Layer 2 Configuration Guide, Cisco IOS XE 17.13.x (Catalyst 9300 Switches) - Configuring Optional Spanning-Tree Features Cisco Catalyst 9300 Series Switches - Cisco

Port Security

The default settings for port security will not age out learned mac addresses. To get aging back to the mac address table default of 5 minutes, set this feature to 5.

Config

interface GigabitEthernet0/0
 switchport access vlan 10
 switchport mode access
 switchport port-security maximum 2
 switchport port-security aging time 5
 switchport port-security
 negotiation auto
 spanning-tree portfast edge

Validation

This is the primary table for this feature. This table is used to populate the mac address table.

switch# show port-security address
               Secure Mac Address Table
-----------------------------------------------------------------------------
Vlan    Mac Address       Type                          Ports   Remaining Age
                                                                   (mins)    
----    -----------       ----                          -----   -------------
  10    5254.000d.6573    SecureDynamic                 Gi0/0       10
-----------------------------------------------------------------------------

Scraping the mac-address table for things programmed in by the port security feature.

switch# show mac address-table secure 
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
  10    5254.000d.6573    STATIC      Gi0/0

Asking port security how many ports are currently controlled by the feature.

switch #show port-security 
Secure Port  MaxSecureAddr  CurrentAddr  SecurityViolation  Security Action
                (Count)       (Count)          (Count)
---------------------------------------------------------------------------
      Gi0/0              2            1                  0         Shutdown
---------------------------------------------------------------------------
Total Addresses in System (excluding one mac per port)     : 0
Max Addresses limit in System (excluding one mac per port) : 4096

References

Cisco - Port Security - IOS-XE 17.14 on the C9300

UDLD

This is the more common way to prevent ULD Failures, enabling ULD.

• ULD: Unidirectional Link Detection.

This feature has two modes:

• Normal: Detect a fiber port that’s miss-cabled. Maybe port 1 and port 2 (four fibers total) got mixed up.

• Aggressive: Detect one way traffic on both fiber, and twisted pair.

From Cisco:

• On fiber-optic or twisted-pair links, one of the ports cannot send or receive traffic.
• On fiber-optic or twisted-pair links, one of the ports is down while the other is up.
• One of the fiber strands in the cable is disconnected.

Requirements

• Both devices need this feature turned on.
• Both sides need the same mode (normal or aggressive)

Global

udld enable

Per Interface

interface 1
  udld enable

Verification

sw1# show udld neighbors 
Port     Device Name   Device ID     Port ID    Neighbor State
----     -----------   ---------     -------    --------------
Gi0/0    91YBLF6S1KI     1            Gi0/0      Bidirectional

Capture

UDLD.pcap

Reference

Layer 2 Configuration Guide, Cisco IOS XE 17.16.x (Catalyst 9500 Switches)

ARP

Captured on-wire.

packet #1 - who has 10.0.0.10? Tell 10.0.0.20
packet #2 - 10.0.0.10 is at ce:b1:5f:58:1d:8a

ARP Request

> Ethernet II

    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: 1a:20:4e:9e:fb:9c (1a:20:4e:9e:fb:9c)
    Type: ARP (0x0806)

> Address Resolution Protocol (request)

    Hardware type: Ethernet (1)
    Protocol type: IPv4 (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: request (1)
    Sender MAC address: 1a:20:4e:9e:fb:9c (1a:20:4e:9e:fb:9c)
    Sender IP address: 10.0.0.20
    Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Target IP address: 10.0.0.10

ARP Reply

> Ethernet II

    Destination: 1a:20:4e:9e:fb:9c (1a:20:4e:9e:fb:9c)
    Source: ce:b1:5f:58:1d:8a (ce:b1:5f:58:1d:8a)
    Type: ARP (0x0806)
    Padding: ... lots of zeros

> Address Resolution Protocol (reply)

    Hardware type: Ethernet (1)
    Protocol type: IPv4 (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: reply (2)
    Sender MAC address: ce:b1:5f:58:1d:8a (ce:b1:5f:58:1d:8a)
    Sender IP address: 10.0.0.10
    Target MAC address: 1a:20:4e:9e:fb:9c (1a:20:4e:9e:fb:9c)
    Target IP address: 10.0.0.20

ARP Attacks

ARP Spoofing and ARP Poisoning are related concepts.

The attacker spoofs an IP on the network, usually the default gateway, with the MAC address of their machine, to intercept all user traffic.

This leads to ARP poisoning where device IP-to-MAC tables now contain false bindings.

Successful ARP attacks lead to traffic hijacking, traffic denial, or man-in-the-middle attacks.

References

ARP spoofing - Wikipedia

DAI

Minimum config

ip dhcp snooping vlan 10
ip arp inspection vlan 10
ip arp inspection validate src-mac dst-mac ip 
!
! Ports
!
interface GigabitEthernet0/1
 description towards DHCP server
 ip arp inspection trust
 ip dhcp snooping trust

Validation

access-1# show ip dhcp snooping binding 
MacAddress          IpAddress        Lease(sec)  Type           VLAN  Interface
------------------  ---------------  ----------  -------------  ----  --------------------
52:54:00:0D:65:73   10.10.10.102     80574       dhcp-snooping   10    GigabitEthernet0/0
Total number of bindings: 1

access-1# show ip arp inspection 

Source Mac Validation      : Enabled
Destination Mac Validation : Enabled
IP Address Validation      : Enabled

 Vlan     Configuration    Operation   ACL Match          Static ACL
 ----     -------------    ---------   ---------          ----------
   10     Enabled          Active                         

 Vlan     ACL Logging      DHCP Logging      Probe Logging
 ----     -----------      ------------      -------------
   10     Deny             Deny              Off          

 Vlan      Forwarded        Dropped     DHCP Drops      ACL Drops
 ----      ---------        -------     ----------      ---------
   10            134              0              0              0

 Vlan   DHCP Permits    ACL Permits  Probe Permits   Source MAC Failures
 ----   ------------    -----------  -------------   -------------------
   10             48              0              0                     0

 Vlan   Dest MAC Failures   IP Validation Failures   Invalid Protocol Data
 ----   -----------------   ----------------------   ---------------------
          
 Vlan   Dest MAC Failures   IP Validation Failures   Invalid Protocol Data
 ----   -----------------   ----------------------   ---------------------
   10                   0                        0                       0

Reference

Cisco - Dynamic ARP Inspection

Practical Networking - Gratuitous ARP

Proxy ARP

There are lots of requirements to get this to work.

• Router needs to see the host on-link
• Host is configured for a much larger subnet then what is actually present
• The ARP target the host is requesting is in the routing table
• The ARP target cannot be out the same interface the ARP was heard on

Lab topology

Evidences

This happens if the host is off-link for the router. The router will see the arp, but also filter it.

*May  4 20:39:31.619: IP ARP req filtered src 192.168.100.100 5254.00b7.f21b, dst 192.168.23.2 0000.0000.0000 wrong cable, interface GigabitEthernet1 tableid 0
*May  4 20:39:32.675: IP ARP req filtered src 192.168.100.100 5254.00b7.f21b, dst 192.168.23.2 0000.0000.0000 wrong cable, interface GigabitEthernet1 tableid 0

Verification

R1# show ip traffic | i proxy
        0 proxy name requests, 0 where-is requests, 0 other
  Sent: 0 address requests, 0 address replies (0 proxy)
        0 proxy name replies, 0 where-is replies
  Sent: 1 requests, 8 replies (2 proxy), 0 reverse

Captures

ARP-proxy-arp.pcap

References

Cisco - Proxy ARP

Cisco - Proxy ARP Problem

Metro Ethernet

General Caveats

• Shut down interfaces before configuring (best practice on ME3400)
• No DTP — dynamic trunk negotiation does not exist
• No VTP — VLAN Trunking Protocol does not exist
• Default STP mode is PVST
• Typically deployed as the last hop (service provider edge / CPE)
• Port model is based on port-types and UNI VLANs rather than standard IOS switchport modes

Port Types (MEF Terminology)

UNI — User Network Interface

• Connects to customer end devices (phones, computers, routers at the customer site)
• All ports except uplinks default to UNI
• Can only forward traffic toward NNI ports
• No STP
• No CDP
• No Link Aggregation

ENI — Enhanced Network Interface

• Connects to routers or switches (slightly trusted devices)
• Supports STP
• Supports CDP
• Supports Link Aggregation

NNI — Network Node Interface

• Network-to-Network — connects to other provider nodes/uplinks
• No Layer 2 protocol filtering — passes BPDUs, CDP, etc. transparently

Port Type Comparison

Private VLANs

Default port type on a Catalyst ME3400 is UNI (User Network Interface)

UNI ports can only send traffic to NNI ports in the same vlan. The default UNI mode is isolated.

show vlan uni-vlan type

show port-type

You can set the uni-vlan type with this command:

vlan 100
  uni-vlan community

This example uses a ME3400.

Config Example

vlan 100
private-vlan primary
private-vlan association 1000,2000,3000
!
vlan 1000
private-vlan isolated
!
vlan 2000
private-vlan community
!
vlan 3000
private-vlan community
!
!
interface FastEthernet0/2
!
! Tell it its a private-vlan host
! Tell it which private VLANs its in
!
 switchport private-vlan host-association 100 1000
 switchport mode private-vlan host
 duplex full
end
!
interface GigabitEthernet0/1
 port-type nni
 switchport private-vlan mapping 100 1000,2000,3000
 switchport mode private-vlan promiscuous
 speed nonegotiate
end

vlan 100
 private-vlan association add ... this doesn't work at all!!

Verification

switch #show vlan private-vlan 

Primary Secondary Type              Ports
------- --------- ----------------- ------------------------------------------
100     1000      isolated          Fa0/1, Fa0/2, Gi0/1
100     2000      community         Fa0/3, Fa0/4, Gi0/1
100     3000      community         Gi0/1

SW1#show vlan private-vlan type 

Vlan Type
---- -----------------
100  primary          
1000 isolated         
2000 community        
3000 community  

MPLS

MPLS Requires CEF

Frame Format

  RFC 3032 - MPLS Label Stack Encoding

  0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |                Label                  | Exp |S|       TTL     | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

                      Label:  Label Value, 20 bits
                      Exp:    Experimental Use, 3 bits
                      S:      Bottom of Stack, 1 bit
                      TTL:    Time to Live, 8 bits

MPLS Control Plane

Local Label - This is what the LSP tells others it wants to recieve.

Outgoing label - What the LSP does next.

Null Labels

[3] Implicit Pop the label. Implicit because the label is missing.

[0] Explicit Keep the label, but the destination must pop it. Used for MPLS QoS.

Steps To Build The LFIB

1. Find The Next-Hop For The Destination Prefix

show ip route 3.3.3.4
 * 10.1.2.1, from 10.1.2.1 via Eth0/0

2. Confirm The LDP Neighbor Behind That Next-Hop

show mpls ldp neighbor 10.1.2.1
 * peer LDP ident 3.3.3.3; Local LDP ident 2.2.2.2

3. Check What Label That Peer Advertised For The Prefix

show mpls ldp binding 3.3.3.4 255.255.255.255
 * remote binding: lsr: 3.3.3.3, label: imp-null

4. Verify The Resulting LFIB Entry

show mpls forwarding 3.3.3.4
- Local 20, Outgoing Pop, Prefix 3.3.3.3/32, Outgoing Int E0/0, Nexthop 10.1.2.1

Testing Pseudowires

You can test MPLS pseduowire labels by doing labeled pings.

ping mpls pseudowire 3.3.3.3 11000

MPLS DiffServ

• Network is a single DiffServ Domain

• ip2mpls Copy DSCP to EXP

• mpls2mpls Copy EXP Upward and Downward

• mpls2ip Copy EXP into the DSCP

• Uniform changes the markings.

Short-Pipe

• Customer network is a different DiffServ Domain
• ip2mpls # Maybe copy
• Copy top EXP into lower labels.
• Use the exposed DSCP when there is no label.
• When mpls2ip is done, act on the ip information not the prior EXP bits.

Pipe Mode (Multiple DiffServ Domains, Default, No Change To CE Marking)

• ip2mpls # Maybe copy DSCP to EXP.
• SP MPLS exp may be remarked in transit
• IPv4 is not remarked at egress
• Customer marking is unchanged.
• mpls2ip operation is based on what the prior MPLS EXP tag was.
• Must shut off PHP

References

RFC 3270: Multi-Protocol Label Switching (MPLS) Support of Differentiated Services | RFC Editor

L3VPN

Customer Exchanges routes with the provider via a routing protocol

VPN just means isolation. No encryption or confidentiality

PE-CE links are their own island

VRF = VPN Routing Forwarding Instance

You can use loopbacks to route between VRFs

L3VPNs rely on Extended Comunnities.

• Basically just arbitrary TLVs attached to BGP prefixes

VPNV4 = PE to PE Label Information ipv4 unicast vrf = BGP within vrfs for PE to CE connectivity

Example

router bgp 100
 neighbor 3.3.3.3 remote-as 100  ! note the neighbor is the same AS, this is the PE
!
address-family vpnv4
 neighbor 3.3.3.3 activate ! this is a global neighbor, part of our network
 neighbor 3.3.3.3 send-community extended
!
address-family ipv4 unicast vrf red
 neighbor 4.4.4.4 remote-as 400 ! Customer neighbor, in their own VRF instance 
 neighbor 4.4.4.4 activate ! this is if you speak BGP to them. It could be any VRF aware IGP. You need to redistribute from the IGP into BGP

Route Distinguishers

• ALL CE routes from ALL VRFs are placed into the same VPNv4 table. What makes them unique to vrfs are the RDs.
  • Transparent to the customer, and only lives on the PE router
  • 100:100:192.168.10/24

Route Targets

• This is the BGP extended community
• route-target export adds the community to the outbound update
• route-target import defines what routes to bring into the VRF

One RD per customer site.

show ip bgp vpnv4 all sum VPNv4 show ip bgp sum IPv4

LDP label gets from PE to PE VPN label identifies VRF to remote PE

PBB

PBB 802.1ah PBB or MAC-in-MAC)

graph LR
    CE1([CE]) ---|UNI .1ad| PEB1[PEB]
    CE2([CE]) ---|UNI .1ad| PEB2[PEB]

    subgraph AL["Access Network · 802.1ad"]
        PEB1 --- PBa1[PB]
        PEB2 --- PBa1
        PBa1 --- PBa3[PB]
        PBa1 --- PBa2
        PBa2[PB] --- PBa3[PB]
    end

    PBa3 ---|UNI .1ah| BEB1
    PBa3 ---|UNI .1ah| BEB2

    subgraph CORE["Core Network · 802.1ah"]
        BEB1[BEB] --- BCB1[BCB]
        BEB2[BEB] --- BCB1
        BCB1 --- BCB2[BCB]
        BCB2 --- BCB3
        BCB1 --- BCB3[BCB]
        BCB3 --- BEB3[BEB]

    end

    BEB3 ---|UNI .1ah| PBb1

    subgraph AR["Access Network · 802.1ad"]
        PBb1[PB] --- PBb3[PB]
        PBb1 --- PBb2
        PBb2[PB] --- PBb3
        PBb3 --- PEB3[PEB]
    end

    PEB3 ---|UNI .1ad| CE3([CE])

Terms

• PB: Provider Bridge.

• PEB: Provider Edge Bridge.

• B-Component: A bridging component contained in a BEB that bridges in provider space (Backbone MAC addresses, B-VLAN).

• BCB: Backbone Core Bridge, An S-VLAN bridge used within the core of a PBBN.

• BEB: Backbone Edge Bridge, A backbone edge bridge positioned at the edge of PBBN that encapsulates customer frames for transmission across a PBBN.

• B-MAC: Backbone MAC Address. An individual MAC address associated with a Provider Instance Port (PIP) and used in creating the MAC header of I-tagged frames transmitted across a PBBN.

• Backbone Service Instance: An instance of the MAC service in a PBBN provided between two or more Virtual Instance Ports (VIPs) in BEBs.

• I-SID: Backbone Service Instance Identifier. A 24-bit field of the backbone service instance tag (I-TAG) that identifies the backbone service instance of a frame. The I-SID defines the service instance that the frame should be “mapped to”.

• I-TAG Backbone Service Instance Tag. A tag with a Ethertype value allocated for IEEE 802.1q backbone service instance tag type.

• B-TAG: Backbone VLAN Tag. A field defined in IEEE 802.1ah provider MAC encapsulation header that conveys the backbone VLAN ID information. The format of the B-TAG is same as that of an IEEE 802.1ad S-TAG field.

• CBP: Customer Backbone Port. A BEB port that can receive and transmit I-tagged frames for multiple customers,

and can assign B-VIDs (backbone VLAN IDs) and translate I-SID on the basis of the received I-SID.

• I-Component: A bridging component contained in a BEB that bridges in the Customer space (Customer MAC addresses, S-VLAN).
• PIP: Provider Instance Port. The set of Virtual Instance Ports (VIPs) that are supported by a single instance.
• Service frame: A frame exchanged between a provider and a customer.
• VIP: Virtual Instance Port. A bridge port on an I-Component in BEB that provides access to a single backbone service instance.

References

L2VPN and Ethernet Services Configuration Guide for Cisco ASR 9000 Series Routers, IOS XR Release 24.1.x, 24.2.x, 24.3.x, 24.4.x - Implementing IEEE 802.1ah Provider Backbone Bridge Cisco ASR 9000 Series Aggregation Services Routers - Cisco

Circuit Emulation

Key Terms

What Is CEM?

CEM treats data as an arbitrary bit stream — the actual Layer 1/Layer 2 format is irrelevant to the transport. This makes it ideal for carrying legacy or opaque traffic over modern packet-switched networks.

Common CEM Use Cases

• 2G / 3G mobile backhaul traffic
• T1 / E1 circuit emulation over packet networks
• PBX-to-PBX connectivity
• Inter-MSC (Mobile Switching Center) connectivity
• Already-encrypted traffic with no defined structure (government, high-security)
• Proprietary synchronous or asynchronous data streams
• Leased line emulation

CEoP SPAs (for Cisco 7600)

Platforms Supporting CEM

Key Configuration Note

Creating a channel-group under a T1 controller automatically creates the associated serial interface.

Clock Distribution

• The hub router owns the clock
• The spoke router recovers the clocks from 0/0/0
• The spoke router uses that clock, to sync lines 0/0/1, 0/1/0, and 0/1/1`

        Hub Router                                                      Spoke Site Router      
                                                                                               
 <Reference Clock>                                                                             
     │                                                                                         
┌────┼─────────────────────┐                                     ┌──────────────────────────┐  
│    ▼                     │                                     │                          │  
│┌────────┬──────► Internal│ 0/0/0─────────────────────────0/0/0 │Line->Network-Clock ──┐   │  
││PLL (1) │                │                                     │                      │   │  
││ Clock  ├──────► Internal│ 0/0/1─────────────────────────0/0/1 │Internal ◄─────────┐  │   │  
│└────┬──┬┘                │                                     │                   │  ▼   │  
│     │  └───────► Internal│ 0/0/2─────────────────────────0/1/0 │Internal ◄─────┬───┴─────┐│  
│     │                    │                                     │               │PLL  (2) ││  
│     └──────────► Internal│ 0/0/3─────────────────────────0/1/1 │Internal ◄─────┤ Clock   ││  
│                          │                                     │               └─────────┘│  
└──────────────────────────┘                                     └──────────────────────────┘  
                    4 Port card in Slot 0     2x 2 port cards Slot 0 and Slot 1        │       
                                                                                       ▼       
 PLL = Phase Locked Loop                                             <clock recovered from hub>

Wireshark Decoding

• SAToP traffic can be decoded using the pwsatopcw protocol keyword
• If Wireshark does not auto-detect the encapsulation, right-click the frame → Decode As and manually select the correct protocol
  • You need to already know what format the traffic is in — there is no auto-detection for pseudowire types

CEM Command Reference

Key Configuration Notes

• TDM byte = one timeslot
• xconnect requires matching VCIDs on both ends to bring up the pseudowire connection

SPAN, RSPAN, ERSPAN

Local

monitor session 1 source interface GigabitEthernet1/0/1 both
monitor session 1 destination interface GigabitEthernet1/0/2

RSPAN

• VLAN Encapsulated
• Does not support layer 2 protocols
  • No CDP, BPDUs, LLDP, etc
• If the source is a trunk port, you can use the filter keyword to select specific vlans

Source switch

vlan 3000
 remote-span
monitor session 1 source interface GigabitEthernet1/0/1 both
monitor session 1 destination remote vlan 3000

Destination switch

vlan 3000
 remote-span
monitor session 1 source remote vlan 3000
monitor session 1 destination interface GigabitEthernet1/0/2

ERSPAN

GRE Encapsulated.

These will encapsulate BPDUs and other Layer 2 protocols.

These need ip routing turned on.

These do not support QoS.

Source switch

monitor session 1 type erspan-source
 !
 ! Could also put a vlan here
 !
 source interface Gi2
 destination
  erspan-id 100
  ip address 10.0.12.2
  origin ip address 10.0.12.1
 no shutdown

Destination switch

monitor session 1 type erspan-destination
 destination interface Gi2
 source
  erspan-id 100
  !
  ! An outside address on this box, not a loopback.
  ! this is the de-encapsulation interface.
  !
  ip address 10.0.12.2
 no shutdown

References

Cisco - Configuring SPAN and RSPAN

Cisco - Configuring ERSPAN

Cisco - ERSPAN Restrictions (IOS XR)

Multicast

Terms

Multicast

• A one-to-many service using UDP packets destined to group IP address
• Hosts subscribe to the group, routers and switches replicate for the group

IGMP — Internet Group Management Protocol

• Receivers send IGMP to LHR to request multicast streams
• Switches see IGMP (for snooping), and the LHR uses this to build the MDT

PIM — Protocol Independent Multicast

• Multicast capable routers communicate to each over via PIM

IIF — Incoming Interface

• AKA, the RPL interface
• Part of the MDT

OIL — Outgoing Interface List

• Part of the MDT

MDT — Multicast Distribution Tree

• The full set of links participating in multicast, via PIM, IGMP, including IILs, and OILs

RP — Rendezvous Point

• A router designated as the root of a shared tree

(*, G)

• Pronounced as “Star comma Gee”
• AKA, a shared tree
• Require a RP
• Called Star comma Gee, because typing “show ip mroute” … this is what shows up

(S, G)

• Pronounced as “Ess comma Gee”
• AKA a source tree. These do not require a RP

Source Tree

• AKA, SPT, or shortest path tree. SPT is best tree.

RPT — Rendezvous Point Tree

• *,G that points towards the RP.

ASM — Any Source Multicast

• The host only knows the group it wants to receive (239.10.10.10)

SSM — Source Specific Multicast

• The host already knows the source, and group address (10.0.0.1, 232.10.10.10)

Upstream

• Towards the source

Downstream

• Towards group members

FHR — First hop router

• This router receives a multicast stream

LHR — Last Hop Router

• Receives IGMP messages from receivers, which are translated into PIM join messages

MRIB — Multicast Routing Information Base

• Shows RPTs, SPTs, RPFIs, OILs, and IILs

MFIB — Multicast Forwarding Information Base

• AKA MFIB
• Used the program the ASICS

RIB — Routing Information Base

DF — Designated Forwarder

• Used in PIDIR-PIM

RPF - Reverse Path Forwarding

PIM is protocol independent, in the sense, that if a stream turns on, it must have a source, so it takes the form (0.0.0.1, 239.1.1.1), a (S,G).

If we do show ip route 10.0.0.1, we’ll see the interface the router intends to send any traffic towards that source address. This is the upstream interface.

As multicast traffic flows from 10.0.0.1, it should flow into the upstream interface, and out of any downstream interfaces the OIL.

Tracing the traffic back to the source this way is called reverse path forwarding and the interface along this path is the RPF.

The PIM neighbor on the RPF is called the RPF neighbor.

Any multi-cast traffic from any given source, not received on the RPF is discarded. This prevents loops.

Shared Trees

(*,G) entries in the mroute table require fewer resources, since multiple sources can use the same tree.

(*,G) entries in the mroute table represent a security risk, because any source can send to this shared tree.

Theory (in V4)

Multicast is always TO a group, a destination, or a set of destinations.

Multicast comes from an older time. Unlike Unicast addresses, you can tell via bits if a v4 address is multicast.

A multicast address always start with 1110

Common L3 Addresses

Same Broadcast Domain

Can Be Forwarded

PIM

PIM forms adjacencies in only one direction

The multicast source is the root of the tree. Packets flow downstream from the source. Control plane traffic like PIM joins flow upstream to the RP, or to the reciever.

IANA Assignments

PIM

PIM Message Types

Shared-Tree (*,G)

• Shared trees are essential for multiple senders to the same group

• A single tree is built for each group, regardless of source

  • 3 sources, 1 tree

• Selects a router as the root of the tree

• If a receiver is on the same subnet as the sending host, it will need to revert to PIM Dense for that segment

• This isn’t always better. Shared trees will typically take suboptimal paths through a network

• Source trees are better distributed, hence they are more robust

• RP Selection is a hassle

Source Based Multicast (S,G)

• PIM dense uses a separate tree for each multicast source and destination group.
• Groups do not share trees.
  • 3 Sources 3 trees.

Commands

show mrib route
show ip mroute
!
! PIM
!
show pim rpf hash
show pim range-list
show pim topology
!
! What interface should I receive this host traffic from?
!
show ip rpf 10.0.0.0
show ip mfib
!
! See if multicast even works
!
show ip pim stats
!
! PIM traffic
!
show ip pim interface detail
!
! DF election
!
show ip pim interface df

FLAGS
 A - Accepting. This interface is accepting data
 F - Forwarding. Where to send multicast traffic

Nexus 7K

show forwarding multicast route group <>

Lab Stuff

BPF - Capture all PIM, but not PIM hello messages.

ip proto 103 and not ether[34] == 0x20

References

IP Multicast - Cisco

Pim Dense

Based on RFC 3973 Protocol Independent Multicast Dense Mode (PIM-DM)

• Push Model

  • Good for when every subnet probably wants this traffic

• No PIM DR

  • All FHR forward multicast traffic
    • Multicast traffic is flooded out every interface that isn’t the RPF.

• Eventually builds a SPT after prunes

• IGMP joins turn into graft messages

• Prunes last 3 minutes

  • Flood and Prune
  • Routers with no Receivers or duplicate S,G traffic prune.
  • 224.0.0.13 to find neighbors
  • Receivers prune back
  • Router attached to LAN listens for multicast control plane.
    • Receives source traffic
      • Insert (*,G) and (S,G) into mrib
      • Incoming traffic is attached to IIL
      • OIL is all other interfaces
      • Flood to OIL
      • PIM dense always uses SPT.

• Prune occurs

  • Traffic flows stop, but (S,G) remains in table
  • Multicast fails RPF
  • No downstream neighbor or reciever
  • Downstream sent prune
  • LAN Prune override exception

• After pruning

  • Flood again, prune back, flood again, prune back

PIM Sparse

Based on RFC4601 - Protocol Independent Multicast Sparse Mode (PIM-SM)

• Explicit joins everywhere. No flooding.
• LHR, sends a PIM-Join towards the RP, building a (*,G).
• Phased *
  1. The RPT tree
  • Receivers sending their (*,G) messages towards the RP.
  • FHR encapsulates the multicast traffic directly towards the RP.
  • PIM-Register
  • RP de-encapsulates the traffic, sending it down the RPT.
  • 2. Register Stop
    • The RP sends a (S,G) towards the source.
    • When multicast packets start showing up, without encapsulation, the RP sends a Register-Stop.
  • 3. SPT tree
    • LHR requests a (S,G) entry towards it’s upstream, until it’s joined to the (S,G) tree.
    • When the LHR starts getting two copies of the traffic, it sends a (S,G,rpt) prune message, towards the RP. (A prune specific to the RPT)
• If two LHRs exist, and duplicate traffic is detected a PIM elections happens.
  • These Asserts are every 3 minutes.
  • RPTbit, 0 is preferred and means “has (S,G) tree”
    • Metric Preference (Administrative Distance)
      • Metric
        • IP address of subnet interface.
• Specify the tunnel, for the pim-register messages on Cisco via ip pim register-source loopback 0
• The tunnel interface encapsulates the entire multicast packet, which adds 28 bytes of overhead. Packets close to the MTU will be silently dropped on IOS-XE.

a DR is elected by highest priority, or highest IP in the subnet.

• DR sends the PIM join upstream.

The RP always gets the stream, even if it has no receivers to forward it to.

Captures

PIM-SM-register-register-stop-prune.pcap

BIDIR-PIM

• Superset of PIM-SM
• No (S,G) entries
• Traffic can flow up and down the same tree.
• Still needs RPs
  • RP must be dedicated to BIDIR-PIM.
• Each bidirectional link has a DF election.
  • Ingress packets on any PIM interface can be forwarded downstream onto DF links.
    • No DF links, no forwarding.
  • Ingress packets to a DF can be forwarded upstream via the RPF towards the RPA.

References

RFC 5015: Bidirectional Protocol Independent Multicast (BIDIR-PIM) | RFC Editor

Auto-RP

Cisco devices can announce their willingness to be an RP, via cisco-rp-announce

A different service, a mapping agent, will read these messages, pick a winner, then advertise that out via cisco-rp-discovery

• 5.5.5.5, Candidate RP.
• 4.4.4.4, mapping agent.

R4# show ip pim autorp

AutoRP Information:

  AutoRP is enabled.
  RP Discovery packet MTU is 1500.
  224.0.1.40 is joined on Loopback0.
  AutoRP groups over sparse mode interface is enabled

PIM AutoRP Statistics: Sent/Received
  RP Announce: 0/16, RP Discovery: 64/42

These packets are slow.

R4# debug ip pim auto-rp

PIM Auto-RP debugging is on
!
! Sent to cisco-rp-discovery
!
*Apr 25 19:57:08.940: Auto-RP(0): Build RP-Discovery packet
*Apr 25 19:57:08.941: Auto-RP(0):  Build mapping (224.0.0.0/4, RP:5.5.5.5), PIMv2 v1,
*Apr 25 19:57:08.942: Auto-RP(0): Send RP-discovery packet of length 48 on GigabitEthernet0/3 (1 RP entries)
*Apr 25 19:57:08.943: Auto-RP(0): Send RP-discovery packet of length 48 on GigabitEthernet0/4 (1 RP entries)
*Apr 25 19:57:08.945: Auto-RP(0): Send RP-discovery packet of length 48 on GigabitEthernet0/0 (1 RP entries)
*Apr 25 19:57:08.948: Auto-RP(0): Send RP-discovery packet of length 48 on Loopback0(*) (1 RP entries)
*Apr 25 19:57:12.008: Auto-RP(0): Received RP-discovery packet of length 48, from 10.0.45.5, ignored
!
! Received by cisco-rp-announce
!
*Apr 25 19:58:30.159: Auto-RP(0): Received RP-announce packet of length 48, from 5.5.5.5, RP_cnt 1, ht 181
*Apr 25 19:58:30.159: (0): pim_add_prm:: 224.0.0.0/240.0.0.0, rp=5.5.5.5, repl = 0, ver =3, is_neg =0, bidir = 0, crp = 0
*Apr 25 19:58:30.160: Auto-RP(0): Update
*Apr 25 19:58:30.160:  prm_rp->bidir_mode = 0 vs bidir = 0 (224.0.0.0/4, RP:5.5.5.5), PIMv2 v1
R4# undebug all
All possible debugging has been turned off

References

Cisco - Configuring a Rendezvous Point

MSDP

Used to link together multicast domains via RPs.

A source turns on, and when the first RP finds out about it, it notifies other RPs.

Details

• RPs register to each other, in different multicast domains.
• RP sends a SA (source active) message.
• Still needs PIM running for the S,G.
• TCP port 639.
• Has keepalives.

show ip msdp peer show ip msdp sa-cache

Startup

1. Multicast source starts up.

2. FHR router sends a register message to the closest RP.

3. RP registers this as a SA (Source Active).

4. Sends Source Active Messages to other RPs

(*,G) means there is an interested receiver.

Config

!
! RP1
!
int lo0
description "Only used for Anycast RP"
ip address 10.0.0.1 255.255.255.255
!
int lo1
ip add 10.1.1.1 255.255.255.255
!
ip msdp peer 10.1.1.2 connect-source loopback 1
ip msdp originator-id loopback 1
ip pim rp-address 10.0.0.1

!
! RP2
!
int lo 0
  description "Only used for Anycast RP"
  ip address 10.0.0.1 255.255.255.255
!
int lo1
  ip address 10.1.1.2 255.255.255.255
!
ip msdp peer 10.1.1.1 connect-source lo1
ip msdp originator-id loopback 1
ip pim rp-address 10.0.0.1

References

RFC 4611 - Multicast Source Discovery Protocol (MSDP) Deployment Scenarios

Multicast L2 Addressing

An L2 multicast address is 6 bytes, or 48 bits.

The address is built by combining these three things.

Multicast Prefix + 0 + low-order-23-bits-from-v4

Multicast Prefix

The Multicast Prefix is 01:00:5E 3 bytes or 24 bits.

25th Bit

A binary zero 0

Last 23 bits

Using 232.10.10.10

Copy the low order 23 bits directly from the v4 address.

232.10.10.10/8

1110 1000 . 0000 1010 . 0000 1010 . 0000 1010
             \______________________________/

000 1010 0000 1010 0000 1010

Building the Address

Using 232.10.10.10

          1 :         0 :        5E :        0A :       0A  :        0A
  0000 0001 . 0000 0000 . 0101 1110 . 0000 1010 . 0000 1010 . 0000 1010
  \__________________________________/|\______________________________/
         Multicast Prefix             |   The 23 bits from the v4 IP
           Always 01:00:5E            |
                                      └─ The required zero bit for multicast

becomes 01:00:5E:0A:0A:0A

Multicast Overlap

To express a v4 multicast address uniquely in L2 we need 28 bits, since the first 4 bits of every v4 multicast address are 1110

L2 in multicast only gets 23 bits, which means 5 bits of overlap.

2^5 is 32, so there are 32 v4 addresses that share the same L2 address.

All 32 IPv4 addresses mapping to 01:00:5E:0A:0A:0A

══════════════════════════════════════════════════════════════════════════════
Address           Octet 1    Octet 2    Octet 3    Octet 4
──────────────────────────────────────────────────────────────────────────────
224. 10.10.10     1110 0000  0000 1010  0000 1010  0000 1010
224.138.10.10     1110 0000  1000 1010  0000 1010  0000 1010
225. 10.10.10     1110 0001  0000 1010  0000 1010  0000 1010
225.138.10.10     1110 0001  1000 1010  0000 1010  0000 1010
226 .10.10.10     1110 0010  0000 1010  0000 1010  0000 1010
226.138.10.10     1110 0010  1000 1010  0000 1010  0000 1010
227 .10.10.10     1110 0011  0000 1010  0000 1010  0000 1010
227.138.10.10     1110 0011  1000 1010  0000 1010  0000 1010
228 .10.10.10     1110 0100  0000 1010  0000 1010  0000 1010
228.138.10.10     1110 0100  1000 1010  0000 1010  0000 1010
229 .10.10.10     1110 0101  0000 1010  0000 1010  0000 1010
229.138.10.10     1110 0101  1000 1010  0000 1010  0000 1010
230 .10.10.10     1110 0110  0000 1010  0000 1010  0000 1010
230.138.10.10     1110 0110  1000 1010  0000 1010  0000 1010
231 .10.10.10     1110 0111  0000 1010  0000 1010  0000 1010
231.138.10.10     1110 0111  1000 1010  0000 1010  0000 1010
232 .10.10.10     1110 1000  0000 1010  0000 1010  0000 1010  < --- This is our SSM address
232.138.10.10     1110 1000  1000 1010  0000 1010  0000 1010
233 .10.10.10     1110 1001  0000 1010  0000 1010  0000 1010  < --- An address in the GLOP block
233.138.10.10     1110 1001  1000 1010  0000 1010  0000 1010
234 .10.10.10     1110 1010  0000 1010  0000 1010  0000 1010
234.138.10.10     1110 1010  1000 1010  0000 1010  0000 1010
235 .10.10.10     1110 1011  0000 1010  0000 1010  0000 1010
235.138.10.10     1110 1011  1000 1010  0000 1010  0000 1010
236 .10.10.10     1110 1100  0000 1010  0000 1010  0000 1010
236.138.10.10     1110 1100  1000 1010  0000 1010  0000 1010
237 .10.10.10     1110 1101  0000 1010  0000 1010  0000 1010
237.138.10.10     1110 1101  1000 1010  0000 1010  0000 1010
238 .10.10.10     1110 1110  0000 1010  0000 1010  0000 1010
238.138.10.10     1110 1110  1000 1010  0000 1010  0000 1010
239 .10.10.10     1110 1111  0000 1010  0000 1010  0000 1010
239.138.10.10     1110 1111  1000 1010  0000 1010  0000 1010  < --- an Organizational scope address
══════════════════════════════════════════════════════════════════════════════
                       ^^^^  ^
                       ||||  |
                       └└└└──└─ I incremented these five bits to show the pattern.

References

RFC 1112: Host extensions for IP multicasting | RFC Editor

OSPF

OSPF is protocol 89.

Terms

IFF — If and only if

LSA — Link State Advertisement

LSDB — Link-State Database

OSPF Process ID

Just where the databases lives. Not transmitted. Allows multiple OSPF processes.

DR — Designated Router

The network vertex for a broadcast or NBMA network. Used to simplify the number of FULL adjacencies.

Advertising Router

The router that created the LSA. The value in this field is the RID.

RID — Router ID

A unique 32-bit number to identify the router in a graph. Doesn’t have to be an IP-the-box, but is usually a loopback.

The Update Rule

A router can only modify an LSA, iff it’s RID is inside the “Advertising Router” field.

LS Sequence

Higher sequence numbers are newer LSAs. The first sequence number in any LSA is 0x80000001

LS Checksum

Used to ensure the LSA was transmitted without corruption. Everything is checked except LS Age.

LS Age

LSAs time out in an hour, and are refreshed every 30 minutes. LSA Age increments when they go through routers.

Packet Types

Hello Packets

These things must match for an adjacency to form

• Subnet
• Subnet mask
• Interface MTU
• Area
• Area flags (NSSA, Stub)
• Is DR/BDR enabled
• Authentication
• Hello time
• Dead time

These must not match

• Router ID

Check with debug ip ospf event

Identical Databases

Each router can perform it’s own SPT via Dijkstra’s algorithm.

LSAs are flooded throughout an area, all routers in the same area should have the same LSAs and same database.

R1# show ip ospf database database-summary  | s Area 0
Area 0 database summary
  LSA Type      Count    Delete   Maxage
  Router        5        0        0       
  Network       5        0        0       
  Summary Net   8        0        0       
  Summary ASBR  2        0        0       
  Type-7 Ext    0        0        0       
    Prefixes redistributed in Type-7  0
  Opaque Link   0        0        0       
  Opaque Area   0        0        0       
  Subtotal      20       0        0

R2# show ip ospf database database-summary | s Area 0
Area 0 database summary
  LSA Type      Count    Delete   Maxage
  Router        5        0        0       
  Network       5        0        0       
  Summary Net   8        0        0       
  Summary ASBR  2        0        0       
  Type-7 Ext    0        0        0       
    Prefixes redistributed in Type-7  0
  Opaque Link   0        0        0       
  Opaque Area   0        0        0       
  Subtotal      20       0        0

Can also check with checksums

show ip ospf | i Checksum

Adjacency State Machine

Routing Hierarchy

OSPF has four levels of routing hierarchy.

The bit E is what makes E1 and E2 routes. The bit being set is an E2 route, which is considered less preferred.

Default Route

OSPF has two ways of originating a default route.

default-information originate if a default route is present.

default-information originate always do it anyway.

Cost

Should be set manually on each node.

The defaults make 100Mbps and above all the same cost.

auto-cost reference-bandwidth 40,000

Area Summary

These will show up as an O IA route in OSPF, and a route-to-null on the ABR.

• Requires a route present in the RIB

v4

router ospf 1
 router-id 2.2.2.2
 area 1 range 10.0.0.0 255.255.224.0

v6

router ospfv3 1
 !
 address-family ipv6 unicast
  area 1 range 2001:DB8::/56

Route-Filtering

You can use the same command to tell the router to … exclude these routes from the backbone, via the not-advertise keyword.

Using Range

The area command is now a route-filter.

v4

router ospf 1
 router-id 2.2.2.2
 area 1 range 10.0.0.0 255.255.224.0 not-advertise

v6

router ospfv3 1
 !
 address-family ipv6 unicast
  area 1 range 2001:DB8::/56 not-advertise
 exit-address-family

Using Filter-Lists

These are a bit harder to use, in and out are inbound and outbound to the area.

For this topology

             Area 0                               Area 1            
                                                               
                                 |           10.0.10.0/24           
                                 |         2001:db8:0:10/64         
                                 |                            +----+
                              +----+       +------------------+ R3 |
+----+                        |    +-------+                  +----+
| R1 +------------------------+ R2 |                        
+----+                        |    +------+     
             10.0.0.0/24      +----+      |                   +----+
           2001:db8:0:0/64       |        +-------------------+ R4 |
                                 |           10.0.20.0/24     +----+
                                 |         2001:db8:0:20/64         

v4

ip prefix-list PREFIX_LIST_LOOPBACK_v4 seq 10 deny 1.1.1.1/32
ip prefix-list PREFIX_LIST_LOOPBACK_v4 seq 20 deny 2.2.2.2/32
ip prefix-list PREFIX_LIST_LOOPBACK_v4 seq 30 deny 3.3.3.3/32
!
router ospf 1
 area 0 filter-list prefix PREFIX_LIST_LOOPBACK_v4 in
 area 1 filter-list prefix PREFIX_LIST_LOOPBACK_v4 in

v6

!
ipv6 prefix-list PREFIX_LIST_v6 seq 10 deny FD::1/128
ipv6 prefix-list PREFIX_LIST_v6 seq 20 deny FD::3/128
ipv6 prefix-list PREFIX_LIST_v6 seq 30 deny FD::4/128
!
router ospfv3 1
 !
 address-family ipv6 unicast
  area 0 filter-list prefix PREFIX_LIST_v6 in
  area 1 filter-list prefix PREFIX_LIST_v6 in

Area Types

No External Network Connections

• Stub: From the RFC, these don’t have LSA-5 in them, so no external routes. A stub gets a default injected.
• Totally Stubby: A Cisco area, This blocks LSA-3, LSA-4, and LSA-5. The only injected LSA is a LSA-3 from the ABR for the default.

External Network connections

• NSSA: From the RFC, this is a stub area with an ASBR. The LSAs within the area are LSA-7, and they get converted to LSA-5 by the ABR.
• Totally Stubby NSSA:, same as above, used to connect an external network, a default is injected as a LSA-3.

References

RFC 2328 - OSPF

OSPF DR Election

DR And BDR

OSPF uses explicit acknowledgments (re-sending the LSAs), so as neighbors and adjacencies grow, the amount of OSPF traffic on a network increases.

A network with six ospf routers forming a full-mesh requires 15 adjacencies.

To mitigate the scaling problem, on broadcast segments OSPF elects a DR, and BDR, to maintain the LSDB.

The RFC calls this a “network vertex”. We can also use the term DR.

• All routers listen for hello on 224.0.0.5
• DR floods LSAs to the routers with 224.0.0.5
• DROTHER talks to the DR/BDR on 224.0.0.6

In the diagram (from the RFC), everything connects to N2, so problem solved.

                                    **FROM**
                +---+      +---+
                |RT3|      |RT4|              |RT3|RT4|RT5|RT6|N2 |
                +---+      +---+        *  ------------------------
                  |    N2    |          *  RT3|   |   |   |   | X |
            +----------------------+    T  RT4|   |   |   |   | X |
                  |          |          O  RT5|   |   |   |   | X |
                +---+      +---+        *  RT6|   |   |   |   | X |
                |RT5|      |RT6|        *   N2| X | X | X | X |   |
                +---+      +---+

                          Broadcast or NBMA networks

See OSPF LSAs to see what the actual contents of the LSAs are.

DR

Forms full adjacencies.

R1# show ip ospf neighbor 

Neighbor ID     Pri   State           Dead Time   Address         Interface
2.2.2.2          50   FULL/BDR        00:00:31    10.0.0.2        Ethernet0/0
3.3.3.3           1   FULL/DROTHER    00:00:37    10.0.0.3        Ethernet0/0
4.4.4.4           1   FULL/DROTHER    00:00:34    10.0.0.4        Ethernet0/0
5.5.5.5           1   FULL/DROTHER    00:00:32    10.0.0.5        Ethernet0/0
6.6.6.6           1   FULL/DROTHER    00:00:31    10.0.0.6        Ethernet0/0

• First router online on the segment is the DR.

Drother

• Only forms full adjacencies with the DR, and BDR.
• When it sends LSAs, sends them to the DR/BDR via 224.0.0.6.

R1# show ip ospf neighbor 

Neighbor ID     Pri   State           Dead Time   Address         Interface
2.2.2.2          50   FULL/BDR        00:00:31    10.0.0.2        Ethernet0/0
3.3.3.3           1   FULL/DROTHER    00:00:37    10.0.0.3        Ethernet0/0
4.4.4.4           1   FULL/DROTHER    00:00:34    10.0.0.4        Ethernet0/0
5.5.5.5           1   FULL/DROTHER    00:00:32    10.0.0.5        Ethernet0/0
6.6.6.6           1   FULL/DROTHER    00:00:31    10.0.0.6        Ethernet0/0

OSPF LSAs

Type 1 - Router

DR

R1# show ip ospf database router 1.1.1.1

            OSPF Router with ID (1.1.1.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 32
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 1.1.1.1
  Advertising Router: 1.1.1.1
  LS Seq Number: 8000007B
  Checksum: 0x1A77
  Length: 36
  Number of Links: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.0.0.1
     (Link Data) Router Interface address: 10.0.0.1
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

DROther

R4#show ip ospf database router 4.4.4.4

            OSPF Router with ID (4.4.4.4) (Process ID 1)

                Router Link States (Area 0)

  LS age: 135
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 4.4.4.4
  Advertising Router: 4.4.4.4
  LS Seq Number: 8000007C
  Checksum: 0x5D18
  Length: 36
  Number of Links: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.0.0.1
     (Link Data) Router Interface address: 10.0.0.4
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

Type 2 - Network

R4# show ip ospf database network 

            OSPF Router with ID (4.4.4.4) (Process ID 1)

                Net Link States (Area 0)

  LS age: 183
  Options: (No TOS-capability, DC)
  LS Type: Network Links
  Link State ID: 10.0.0.1 (address of Designated Router)
  Advertising Router: 1.1.1.1
  LS Seq Number: 80000002
  Checksum: 0x4481
  Length: 48
  Network Mask: /24
        Attached Router: 1.1.1.1
        Attached Router: 2.2.2.2
        Attached Router: 3.3.3.3
        Attached Router: 4.4.4.4
        Attached Router: 5.5.5.5
        Attached Router: 6.6.6.6

OSPF LSA Types

LSA Type to Route Type

Format

The Router ID is what is used to build the SPT. It’s very important it’s both

• Correct
• Easy to identify the router

  +-------------------------+ Three fields to differentiate LSAs
  |         LS Age          |     - LS Type
  +-------------------------+     - Link State ID
  |  Options      LS Type   |     - Advertising Router
  +-------------------------+
  |     Link State ID       |  < -- Unique number from the Advertising Router for Each LSA
  +-------------------------+
  |   Advertising Router    |  < -- Router ID
  +-------------------------+
  |    LS Sequence Number   |  < -- How old the LSA is. LSAs with higher numbers are updates to older LSAs
  +-------------------------+
  |      LS Checksum        |
  +-------------------------+
  |        Length           |
  +-------------------------+

OSPF Type 1 LSA - Router

This LSA describes the networks attached to a router.

Topology

                                                 ASBR                                    
                                                  OSPF                                   
                     A23  Regular               ┌────┐     ┌─────┐  EIGRP AS 33  ┌─────┐ 
               ┌────────────────────────────────┤ R3 ├─────┤ R33 ├───────────────┤ R34 │ 
               │                                └────┘     └─────┘ 10.33.34.0/24 └─────┘ 
               │     A24  Stub                  ┌────┐     ┌─────┐                       
               │ ┌──────────────────────────────┤ R4 ├─────┤ R44 │                       
               │ │                              └────┘     └─────┘                       
┌────┐ Area 0 ┌┴─┴─┐ A25  NSSA                  ┌────┐     ┌─────┐  EIGRP AS 55  ┌─────┐ 
│ R1 ├────────┤ R2 ├────────────────────────────┤ R5 ├─────┤ R55 ├───────────────┤ R56 │ 
└────┘        │ ABR│                            └────┘     └─────┘ 10.55.56.0/24 └─────┘ 
              └┬─┬─┘ A26  Totally Stubby        ┌────┐     ┌─────┐                       
               │ └──────────────────────────────┤ R6 ├─────┤ R66 │                       
               │                                └────┘     └─────┘                       
               │     A27  NSSA Totally Stubby   ┌────┐     ┌─────┐  EIGRP AS 77  ┌─────┐ 
               └────────────────────────────────┤ R7 ├─────┤ R77 ├───────────────┤ R78 │ 
                                                └────┘     └─────┘ 10.77.78.0/24 └─────┘ 

Example

R1# show ip ospf database router 

            OSPF Router with ID (10.0.0.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 325
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 10.0.0.1
  Advertising Router: 10.0.0.1
  LS Seq Number: 80000012
  Checksum: 0xDC6
  Length: 48
  Number of Links: 2

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.0.0.1
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.1.2.1
     (Link Data) Router Interface address: 10.1.2.1
      Number of MTID metrics: 0
       TOS 0 Metrics: 1


  LS age: 134
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 10.0.0.2
  Advertising Router: 10.0.0.2
  LS Seq Number: 80000014
  Checksum: 0x26A4
  Length: 48
  Area Border Router
  AS Boundary Router
  Number of Links: 2

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.0.0.2
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.1.2.1
     (Link Data) Router Interface address: 10.1.2.2
      Number of MTID metrics: 0
       TOS 0 Metrics: 1

OSPF Type 2 LSA - Network

This LSA describes a network vertex, a DR for a Broadcast Segment.

Topology

                                                 ASBR                                    
                                                  OSPF                                   
                     A23  Regular               ┌────┐     ┌─────┐  EIGRP AS 33  ┌─────┐ 
               ┌────────────────────────────────┤ R3 ├─────┤ R33 ├───────────────┤ R34 │ 
               │                                └────┘     └─────┘ 10.33.34.0/24 └─────┘ 
               │     A24  Stub                  ┌────┐     ┌─────┐                       
               │ ┌──────────────────────────────┤ R4 ├─────┤ R44 │                       
               │ │                              └────┘     └─────┘                       
┌────┐ Area 0 ┌┴─┴─┐ A25  NSSA                  ┌────┐     ┌─────┐  EIGRP AS 55  ┌─────┐ 
│ R1 ├────────┤ R2 ├────────────────────────────┤ R5 ├─────┤ R55 ├───────────────┤ R56 │ 
└────┘        │ ABR│                            └────┘     └─────┘ 10.55.56.0/24 └─────┘ 
              └┬─┬─┘ A26  Totally Stubby        ┌────┐     ┌─────┐                       
               │ └──────────────────────────────┤ R6 ├─────┤ R66 │                       
               │                                └────┘     └─────┘                       
               │     A27  NSSA Totally Stubby   ┌────┐     ┌─────┐  EIGRP AS 77  ┌─────┐ 
               └────────────────────────────────┤ R7 ├─────┤ R77 ├───────────────┤ R78 │ 
                                                └────┘     └─────┘ 10.77.78.0/24 └─────┘ 

Example

R1# show ip ospf database network 

            OSPF Router with ID (10.0.0.1) (Process ID 1)

                Net Link States (Area 0)

  LS age: 508
  Options: (No TOS-capability, DC)
  LS Type: Network Links
  Link State ID: 10.1.2.1 (address of Designated Router)
  Advertising Router: 10.0.0.1
  LS Seq Number: 80000008
  Checksum: 0x5DA5
  Length: 32
  Network Mask: /24
        Attached Router: 10.0.0.1
        Attached Router: 10.0.0.2

OSPF Type 3 LSA - Summary

These describe networks, reachable via an ABR.

Topology

                                                 ASBR                                    
                                                  OSPF                                   
                     A23  Regular               ┌────┐     ┌─────┐  EIGRP AS 33  ┌─────┐ 
               ┌────────────────────────────────┤ R3 ├─────┤ R33 ├───────────────┤ R34 │ 
               │                                └────┘     └─────┘ 10.33.34.0/24 └─────┘ 
               │     A24  Stub                  ┌────┐     ┌─────┐                       
               │ ┌──────────────────────────────┤ R4 ├─────┤ R44 │                       
               │ │                              └────┘     └─────┘                       
┌────┐ Area 0 ┌┴─┴─┐ A25  NSSA                  ┌────┐     ┌─────┐  EIGRP AS 55  ┌─────┐ 
│ R1 ├────────┤ R2 ├────────────────────────────┤ R5 ├─────┤ R55 ├───────────────┤ R56 │ 
└────┘        │ ABR│                            └────┘     └─────┘ 10.55.56.0/24 └─────┘ 
              └┬─┬─┘ A26  Totally Stubby        ┌────┐     ┌─────┐                       
               │ └──────────────────────────────┤ R6 ├─────┤ R66 │                       
               │                                └────┘     └─────┘                       
               │     A27  NSSA Totally Stubby   ┌────┐     ┌─────┐  EIGRP AS 77  ┌─────┐ 
               └────────────────────────────────┤ R7 ├─────┤ R77 ├───────────────┤ R78 │ 
                                                └────┘     └─────┘ 10.77.78.0/24 └─────┘ 

Example

R1# show ip ospf database summary 

            OSPF Router with ID (10.0.0.1) (Process ID 1)

                Summary Net Link States (Area 0)

  LS age: 1901
  Options: (No TOS-capability, DC, Upward)
  LS Type: Summary Links(Network)
  Link State ID: 10.0.0.3 (summary Network Number)
  Advertising Router: 10.0.0.2
  LS Seq Number: 80000009
  Checksum: 0x8693
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2 

OSPF Type 4 LSA - ASBR Summary

This LSA describes how to reach an ASBR, via an ABR.

Topology

                                                 ASBR                                    
                                                  OSPF                                   
                     A23  Regular               ┌────┐     ┌─────┐  EIGRP AS 33  ┌─────┐ 
               ┌────────────────────────────────┤ R3 ├─────┤ R33 ├───────────────┤ R34 │ 
               │                                └────┘     └─────┘ 10.33.34.0/24 └─────┘ 
               │     A24  Stub                  ┌────┐     ┌─────┐                       
               │ ┌──────────────────────────────┤ R4 ├─────┤ R44 │                       
               │ │                              └────┘     └─────┘                       
┌────┐ Area 0 ┌┴─┴─┐ A25  NSSA                  ┌────┐     ┌─────┐  EIGRP AS 55  ┌─────┐ 
│ R1 ├────────┤ R2 ├────────────────────────────┤ R5 ├─────┤ R55 ├───────────────┤ R56 │ 
└────┘        │ ABR│                            └────┘     └─────┘ 10.55.56.0/24 └─────┘ 
              └┬─┬─┘ A26  Totally Stubby        ┌────┐     ┌─────┐                       
               │ └──────────────────────────────┤ R6 ├─────┤ R66 │                       
               │                                └────┘     └─────┘                       
               │     A27  NSSA Totally Stubby   ┌────┐     ┌─────┐  EIGRP AS 77  ┌─────┐ 
               └────────────────────────────────┤ R7 ├─────┤ R77 ├───────────────┤ R78 │ 
                                                └────┘     └─────┘ 10.77.78.0/24 └─────┘ 

Example

R1# show ip ospf database asbr-summary 

            OSPF Router with ID (10.0.0.1) (Process ID 1)

                Summary ASB Link States (Area 0)

  LS age: 429
  Options: (No TOS-capability, DC, Upward)
  LS Type: Summary Links(AS Boundary Router)
  Link State ID: 10.0.0.33 (AS Boundary Router address)
  Advertising Router: 10.0.0.2
  LS Seq Number: 80000008
  Checksum: 0x4DAE
  Length: 28
  Network Mask: /0
        MTID: 0         Metric: 2 

Type 5 LSA - External

This LSA describes external networks.

Topology

                                                 ASBR                                    
                                                  OSPF                                   
                     A23  Regular               ┌────┐     ┌─────┐  EIGRP AS 33  ┌─────┐ 
               ┌────────────────────────────────┤ R3 ├─────┤ R33 ├───────────────┤ R34 │ 
               │                                └────┘     └─────┘ 10.33.34.0/24 └─────┘ 
               │     A24  Stub                  ┌────┐     ┌─────┐                       
               │ ┌──────────────────────────────┤ R4 ├─────┤ R44 │                       
               │ │                              └────┘     └─────┘                       
┌────┐ Area 0 ┌┴─┴─┐ A25  NSSA                  ┌────┐     ┌─────┐  EIGRP AS 55  ┌─────┐ 
│ R1 ├────────┤ R2 ├────────────────────────────┤ R5 ├─────┤ R55 ├───────────────┤ R56 │ 
└────┘        │ ABR│                            └────┘     └─────┘ 10.55.56.0/24 └─────┘ 
              └┬─┬─┘ A26  Totally Stubby        ┌────┐     ┌─────┐                       
               │ └──────────────────────────────┤ R6 ├─────┤ R66 │                       
               │                                └────┘     └─────┘                       
               │     A27  NSSA Totally Stubby   ┌────┐     ┌─────┐  EIGRP AS 77  ┌─────┐ 
               └────────────────────────────────┤ R7 ├─────┤ R77 ├───────────────┤ R78 │ 
                                                └────┘     └─────┘ 10.77.78.0/24 └─────┘ 

Examples

R1#show ip ospf database external 10.55.56.0

            OSPF Router with ID (10.0.0.1) (Process ID 1)

                Type-5 AS External Link States

  LS age: 353
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 10.55.56.0 (External Network Number )
  Advertising Router: 10.0.0.2
  LS Seq Number: 80000008
  Checksum: 0x86FB
  Length: 36
  Network Mask: /24
        Metric Type: 2 (Larger than any link state path)
        MTID: 0 
        Metric: 100 
        Forward Address: 10.0.0.55
        External Route Tag: 0

OSPFv2 Network and LSA Chart

v3.1 (9-June-2026)

OSPFv2 LSA Default Routes

R2 is advertising a default route to these routers.

Each router is inside a different area type.

                    Regular              ┌────┐
              ┌──────────────────────────┤ R3 │ Type 5
              │                          └────┘
              │     Stub                 ┌────┐
              │ ┌────────────────────────┤ R4 │ Type 3
      Area 0  │ │                        └────┘
┌────┐       ┌┴─┴─┐ NSSA                 ┌────┐
│ R1 ├───────┤ R2 ├──────────────────────┤ R5 │ Type 7
└────┘       └┬─┬─┘                      └────┘
              │ │   Totally Stubby       ┌────┐
              │ └────────────────────────┤ R6 │ Type 3
              │                          └────┘
              │     NSSA Totally Stubby  ┌────┐
              └──────────────────────────┤ R7 │ Type 3
                                         └────┘

Type 3 Default

R2# show ip ospf database summary 0.0.0.0 self-originate 

            OSPF Router with ID (2.2.2.2) (Process ID 1)

                Summary Net Link States (Area 24)

  LS age: 951
  Options: (No TOS-capability, DC, Upward)
  LS Type: Summary Links(Network)
  Link State ID: 0.0.0.0 (summary Network Number)
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000002
  Checksum: 0x73C1
  Length: 28
  Network Mask: /0
        MTID: 0         Metric: 1 


                Summary Net Link States (Area 26)

  LS age: 951
  Options: (No TOS-capability, DC, Upward)
  LS Type: Summary Links(Network)
  Link State ID: 0.0.0.0 (summary Network Number)
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000002
  Checksum: 0x73C1
  Length: 28
  Network Mask: /0
        MTID: 0         Metric: 1 


                Summary Net Link States (Area 27)

  LS age: 953
  Options: (No TOS-capability, DC, Upward)
  LS Type: Summary Links(Network)
  Link State ID: 0.0.0.0 (summary Network Number)
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000002
  Checksum: 0xFA32
  Length: 28
  Network Mask: /0
        MTID: 0         Metric: 1

Type 5 Default

R2# show ip ospf database external 0.0.0.0 self-originate 

            OSPF Router with ID (2.2.2.2) (Process ID 1)

                Type-5 AS External Link States

  LS age: 59
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 0.0.0.0 (External Network Number )
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000001
  Checksum: 0xFEAB
  Length: 36
  Network Mask: /0
        Metric Type: 2 (Larger than any link state path)
        MTID: 0 
        Metric: 1 
        Forward Address: 0.0.0.0
        External Route Tag: 1

Type 7 Default

R2# show ip ospf database nssa-external 0.0.0.0 self-originate

            OSPF Router with ID (2.2.2.2) (Process ID 1)

                Type-7 AS External Link States (Area 25)

  LS age: 312
  Options: (No TOS-capability, No Type 7/5 translation, DC, Upward)
  LS Type: AS External Link
  Link State ID: 0.0.0.0 (External Network Number )
  Advertising Router: 2.2.2.2
  LS Seq Number: 80000001
  Checksum: 0xD0D8
  Length: 36
  Network Mask: /0
        Metric Type: 2 (Larger than any link state path)
        MTID: 0 
        Metric: 1 
        Forward Address: 0.0.0.0
        External Route Tag: 0

OSPF Network Types

OSPF Representation of routers and networks

Moy                         Standards Track                    [Page 13]

RFC 2328                     OSPF Version 2                   April 1998

                                                  **FROM**

                                           *      |RT1|RT2|
                +---+Ia    +---+           *   ------------
                |RT1|------|RT2|           T   RT1|   | X |
                +---+    Ib+---+           O   RT2| X |   |
                                           *    Ia|   | X |
                                           *    Ib| X |   |

                     Physical point-to-point networks


                                                  **FROM**
                      +---+                *
                      |RT7|                *      |RT7| N3|
                      +---+                T   ------------
                        |                  O   RT7|   |   |
            +----------------------+       *    N3| X |   |
                       N3                  *

                              Stub networks

                                                  **FROM**
                +---+      +---+
                |RT3|      |RT4|              |RT3|RT4|RT5|RT6|N2 |
                +---+      +---+        *  ------------------------
                  |    N2    |          *  RT3|   |   |   |   | X |
            +----------------------+    T  RT4|   |   |   |   | X |
                  |          |          O  RT5|   |   |   |   | X |
                +---+      +---+        *  RT6|   |   |   |   | X |
                |RT5|      |RT6|        *   N2| X | X | X | X |   |
                +---+      +---+

                          Broadcast or NBMA networks

References

RFC 2328 - OSPF

OSPFv2 Point-to-Multipoint

p2mp — point-to-multipoint

• Usually Hub-and-Spoke topologies
• No DR election for this network type
• Install a /32

What is Hub and Spoke

This network type assumes that spoke sites cannot reach each other directly, but are in the same L3 subnet.

This network type also assumes hair-pins are OK (traffic must go in and out of the same interface on one node)

Since each router is in the same L3 subnet, but not directly L2 reachable, each router will advertise it’s own, in subnet, IP as a /32.

The use-case for this network type is frame-relay, older network types where traveling through the hub is required.

Multicast

If multicast works (even carried by mGRE) usepoint-to-multipoint.

If multicast does not work, use point-to-point non-broadcast and manually define the neighbors under OSPF.

An Example Problem

A network is set up on Ethernet.

… Maybe it looks like this.

       10.12.34.0/24      
                          
┌──────┐          ┌──────┐
│  R1  ├─┐      ┌─┤  R2  │
└──────┘ │      │ └──────┘
        ┌┴──────┴┐        
        │ Switch │        
        └┬──────┬┘        
┌──────┐ │      │ ┌──────┐
│  R3  ├─┘      └─┤  R4  │
└──────┘          └──────┘

This will not work.

IPs are

• 10.12.34.1
• 10.12.34.2
• 10.12.34.3
• 10.12.34.4

RIDs are

• 1.1.1.1
• 2.2.2.2
• 3.3.3.3
• 4.4.4.4

OSPF Network Types, attempted.

• point-to-multipoint
• point-to-multipoint non-broadcast

R1 - Doesn’t Form a Full Mesh

4.4.4.4 is broken.

Technically it flaps, going up and down.

R1# show ip ospf neighbor 

Neighbor ID     Pri   State           Dead Time   Address         Interface
4.4.4.4           0   EXSTART/  -     00:01:59    10.12.34.4      Ethernet0/0
2.2.2.2           0   FULL/  -        00:01:59    10.12.34.2      Ethernet0/0
3.3.3.3           0   FULL/  -        00:01:45    10.12.34.3      Ethernet0/0

R1 OSPF Debugs Show Adjacency Problems

• R1-R4 Hello messages seem to work fine
• DBD messages from R1 never get to R4.

debug ip ospf adjacency
debug ip ospf hello

R1# show debug
Packet Infra debugs:

Ip Address                                               Port
------------------------------------------------------|----------

OSPF:
  OSPF hello debugging is on
  OSPF adjacency debugging is on

R1#
*Jun  2 23:40:18.278: OSPF-1 HELLO Et0/0: Rcv hello from 4.4.4.4 area 0 10.12.34.4
*Jun  2 23:40:18.278: OSPF-1 ADJ   Et0/0: 2 Way Communication to 4.4.4.4, state 2WAY
*Jun  2 23:40:18.278: OSPF-1 ADJ   Et0/0: Nbr 4.4.4.4: Prepare dbase exchange
*Jun  2 23:40:18.278: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:18.397: OSPF-1 HELLO Et0/0: Rcv hello from 3.3.3.3 area 0 10.12.34.3
*Jun  2 23:40:22.784: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:22.784: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [1]
*Jun  2 23:40:25.728: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.4 area 0 from 10.12.34.1
*Jun  2 23:40:25.728: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.2 area 0 from 10.12.34.1
*Jun  2 23:40:25.728: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.3 area 0 from 10.12.34.1
*Jun  2 23:40:27.481: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:27.481: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [2]
*Jun  2 23:40:28.018: OSPF-1 HELLO Et0/0: Rcv hello from 2.2.2.2 area 0 10.12.34.2
*Jun  2 23:40:32.481: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:32.481: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [3]
*Jun  2 23:40:37.083: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:37.083: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [4]
*Jun  2 23:40:41.603: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:41.603: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [5]
*Jun  2 23:40:45.534: OSPF-1 HELLO Et0/0: Rcv hello from 3.3.3.3 area 0 10.12.34.3
*Jun  2 23:40:46.486: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:46.486: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [6]
*Jun  2 23:40:47.070: OSPF-1 HELLO Et0/0: Rcv hello from 4.4.4.4 area 0 10.12.34.4
*Jun  2 23:40:51.106: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:51.106: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [7]
*Jun  2 23:40:53.805: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.4 area 0 from 10.12.34.1
*Jun  2 23:40:53.805: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.2 area 0 from 10.12.34.1
*Jun  2 23:40:53.805: OSPF-1 HELLO Et0/0: Send hello to 10.12.34.3 area 0 from 10.12.34.1
*Jun  2 23:40:55.769: OSPF-1 ADJ   Et0/0: Send DBD to 4.4.4.4 seq 0xB07 opt 0x52 flag 0x7 len 32
*Jun  2 23:40:55.769: OSPF-1 ADJ   Et0/0: Retransmitting DBD to 4.4.4.4 [8]
*Jun  2 23:40:55.909: OSPF-1 HELLO Et0/0: Rcv hello from 2.2.2.2 area 0 10.12.34.2

R1 - Routing Table is Weird

R1 thinks the way to get to R4 is via R2.

Notice the other OSPF routers are fine.

R1# show ip route

[output omitted]

      10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks
C        10.12.34.0/24 is directly connected, Ethernet0/0
L        10.12.34.1/32 is directly connected, Ethernet0/0
O        10.12.34.2/32 [110/10] via 10.12.34.2, 00:35:13, Ethernet0/0
O        10.12.34.3/32 [110/10] via 10.12.34.3, 00:35:12, Ethernet0/0
O        10.12.34.4/32 [110/20] via 10.12.34.2, 00:34:19, Ethernet0/0

R1 ARP is Correct.

L2 programming for a spoke site doesn’t matter for Hub-and-Spoke designs.

R1# show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.12.34.1              -   aabb.cc00.3d00  ARPA   Ethernet0/0
Internet  10.12.34.2            128   aabb.cc00.4d00  ARPA   Ethernet0/0
Internet  10.12.34.3            128   aabb.cc00.4e00  ARPA   Ethernet0/0
Internet  10.12.34.4            128   aabb.cc00.3e00  ARPA   Ethernet0/0

R1 - CEF Is Wrong

This is how we know something is very wrong, a paradigm has been broken.

CEF is using the next-hop of R2.

R1# show ip cef 10.12.34.4 detail 
10.12.34.4/32, epoch 0
  Adj source: IP adj out of Ethernet0/0, addr 10.12.34.4 7162A5807E90
    Dependent covered prefix type adjfib, cover 10.12.34.0/24
  nexthop 10.12.34.2 Ethernet0/0

Checking Router LSAs

Self-Originated Router-LSA on R4

Point-to-Multipoint describes the network as a series of /32 links. What does that mean?

It means when a router goes FULL the first thing it does is advertise its own connected IP directly into OSPF as a stub Network.

R4# show ip ospf database router self-originate 

            OSPF Router with ID (4.4.4.4) (Process ID 1)

                Router Link States (Area 0)

  LS age: 792
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 4.4.4.4
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000016
  Checksum: 0x9FC5
  Length: 48
  Number of Links: 2

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 2.2.2.2
     (Link Data) Router Interface address: 10.12.34.4
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.12.34.4
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 0

R4 only has adjacency with R2.

R4# show ip ospf neighbor 

Neighbor ID     Pri   State           Dead Time   Address         Interface
N/A               0   DOWN/  -           -        10.12.34.3      Ethernet0/0
1.1.1.1           0   INIT/  -        00:01:59    10.12.34.1      Ethernet0/0
2.2.2.2           0   FULL/  -        00:01:53    10.12.34.2      Ethernet0/0

R2 has an adjacency with R1

R2# show ip ospf neighbor 

Neighbor ID     Pri   State           Dead Time   Address         Interface
1.1.1.1           0   FULL/  -        00:01:38    10.12.34.1      Ethernet0/0
3.3.3.3           0   EXSTART/  -     00:01:51    10.12.34.3      Ethernet0/0
4.4.4.4           0   FULL/  -        00:01:49    10.12.34.4      Ethernet0/0

Checking LSAs for the area.

Notes are embedded in the output.

R1# show ip ospf database router 

            OSPF Router with ID (1.1.1.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 1159
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 1.1.1.1
  Advertising Router: 1.1.1.1
  LS Seq Number: 80000010
  Checksum: 0xCB60
  Length: 60
  Number of Links: 3
!
! R1 is adjacent to R2
!
    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 2.2.2.2
     (Link Data) Router Interface address: 10.12.34.1
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

[output omitted]

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.12.34.1
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 0


  LS age: 1155
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 2.2.2.2
  Advertising Router: 2.2.2.2
  LS Seq Number: 8000001A
  Checksum: 0xE531
  Length: 60
  Number of Links: 3

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 1.1.1.1
     (Link Data) Router Interface address: 10.12.34.2
      Number of MTID metrics: 0
       TOS 0 Metrics: 10
!
! R2 is adjacency to R4
!
    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 4.4.4.4
     (Link Data) Router Interface address: 10.12.34.2
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.12.34.2
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 0

[output omitted]
!
! R4 advertises it's IP as a /32, via p2mp
!
  LS age: 1141
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 4.4.4.4
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000016
  Checksum: 0x9FC5
  Length: 48
  Number of Links: 2

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 2.2.2.2
     (Link Data) Router Interface address: 10.12.34.4
      Number of MTID metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.12.34.4 
     (Link Data) Network Mask: 255.255.255.255
      Number of MTID metrics: 0
       TOS 0 Metrics: 0

The Capture

OSPFv2-p2mp-failing

R1 sends a unicast DBD packet to R4, but uses the wrong mac address.

          10.12.34.0/24         
                                
    .3d00             .4d00     
   ┌──────┐          ┌──────┐   
.1 │  R1  ├─┐      ┌─┤  R2  │ .2
   └──────┘ │      │ └──────┘   
           ┌┴──────┴┐           
           │ Switch │           
           └┬──────┬┘           
   ┌──────┐ │      │ ┌──────┐   
.3 │  R3  ├─┘      └─┤  R4  │ .4
   └──────┘          └──────┘   
    .4e00             .3e00     

The Packets in Text

!
! Send a DBD Packet
!
Frame 1: Packet, 78 bytes on wire (624 bits), 78 bytes captured (624 bits)
Ethernet II, Src: aa:bb:cc:00:3d:00 (aa:bb:cc:00:3d:00), Dst: aa:bb:cc:00:4d:00 (aa:bb:cc:00:4d:00)
Internet Protocol Version 4, Src: 10.12.34.1, Dst: 10.12.34.4
Open Shortest Path First
    OSPF Header
    OSPF DB Description
    OSPF LLS Data Block
!
! Get an ICMP TTL exceeded
!
Frame 2: Packet, 70 bytes on wire (560 bits), 70 bytes captured (560 bits)
Ethernet II, Src: aa:bb:cc:00:4d:00 (aa:bb:cc:00:4d:00), Dst: aa:bb:cc:00:3d:00 (aa:bb:cc:00:3d:00)
Internet Protocol Version 4, Src: 10.12.34.2, Dst: 10.12.34.1
Internet Control Message Protocol
    Type: Time-to-live exceeded (11)
    Code: 0 (Time to live exceeded in transit)
    Checksum: 0xf0db [correct]
    [Checksum Status: Good]
    Unused: 00000000
    Internet Protocol Version 4, Src: 10.12.34.1, Dst: 10.12.34.4
    Open Shortest Path First
        OSPF Header
            Version: 2
            Message Type: DB Description (2)
            Packet Length: 32
            Source OSPF Router: 1.1.1.1

The OSPFv2 Shortest Path Topology.

OSPF using point-to-multipoint assumes the topology looks like this.

         ┌──────┐              
         │  R2  │              
         └──┬───┘              
            │                  
            │                  
          .-~~~-.              
  .- ~ ~-(       )_ _          
 /                    ~ -.     
|                          ',  
 \                         .'  
   ~- ._ ,. ,.,.,., ,.. -~     
     │     '       '    │      
     │                  │      
     │                  │      
┌────┴─┐              ┌─┴────┐ 
│  R1  │              │  R4  │ 
└──────┘              └──────┘ 

Workarounds

You can just make the routers not advertise their /32 networks using prefix suppression. This may not work in all IOS, IOS-XE versions.

interface Ethernet0/0
 ip address 10.12.34.1 255.255.255.0
 ip ospf network point-to-multipoint non-broadcast
 ip ospf prefix-suppression
 ip ospf 1 area 0
end

References

Point-to-multipoint communication - Wikipedia

Solved: OSPF Point-to-multipoint network type - Cisco Community

RFC 2328: OSPF Version 2 | RFC Editor

Cisco - IP Routing: OSPF Configuration Guide - OSPF Mechanism to Exclude Connected IP Prefixes from LSA Advertisements Cisco - IOS 15.2E

OSPF LFA

Relies on Triangles.

References

Loop-Free Alternate: OSPF Meets EIGRP « ipSpace.net blog

RFC 5286: Basic Specification for IP Fast Reroute: Loop-Free Alternates | RFC Editor

OSPF Sham Links

The Problem

A customer with L3VPN service via OSPF-BGP-VPNv4 decides to connect two sites together via OSPF backdoor, a direct connection they manage themselves.

When they turn on their private OSPF peering, all the traffic between these two sites now prefers the new link, vs the L3VPN cloud.

The Solution

Sham links are needed because the routes provided by an L3VPN are O IA. When the OSPF backdoor link comes up it will be preferred for two reasons:

• OSPF has a lower AD than BGP
• O routes are preferred over O IA

A sham link makes two PE routers at different sites in the same customer VRF form an intra-area connection.

From OSPF Sham-Link Support for MPLS VPN - Cisco.

Before you create a sham-link between PE routers in an MPLS VPN, you must:

• Configure a new interface with a /32 address on the remote PE so that OSPF packets can be sent over the VPN backbone to the remote end of the sham-link. The /32 address must meet the following criteria:
  • Belong to a VRF
  • Not be advertised by OSPF
  • Be advertised by BGP
  • You can use the /32 address for other sham-links

References

What is OSPF Sham Links? how to configure OSPF Sham Links? - Cisco Community

EIGRP

Terms

Successor route

• The current best path, with the smallest metric. The “successful” route.

Successor

• The first next-hop router for the successor route.

Feasible distance (FD)

• Lowest metric to reach a subnet. The sum of the RD + local cost.

Reported distance (RD)

• The metric inside a route update from another router. The sending router included it’s FD, which becomes out RD.

Feasibility condition

• If another path is actually a backup, the RD will be less than the current FD.

Feasible successor

• A route that satisfies the feasibility condition and is maintained as a backup route.

Split Horizon

• Never advertise a network, out the same interface it was learned on.

Poison Reverse

• If you must advertise a network out the same interface it was received on, advertise the delay as infinity.

Feasible Successor Algorithm

Topology

┌────────┐            1000             ┌────────┐    10.0.0.0/24
│   R1   ├─────────────────────────────┤   R2   ├──────────────────
└─────┬──┘                             └─┬──────┘      2000
      │            ┌────────┐            │
      └────────────┤   R3   ├────────────┘
         50        └────────┘      50

R2 sends an update

• 10.0.0.0/24 - RD is 2000

R3 Sends an update

• 10.0.0.0/24 - RD is 2050

R1 calculates total path metric.

• R2 is 2000 + 1000 = 3000.
• R3 is 2050 + 50 = 2100. < - Successor route.

Results

• R1 installs the successor route as R1-R2
• R1 picks R1-R3 as the feasible successor because the RD (2050) is less than the FD

Results in the CLI

R1# show ip eigrp topology 10.0.0.0/24
EIGRP-IPv4 Topology Entry for AS(1)/ID(1.1.1.1) for 10.0.0.0/24
  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 2100
P 10.0.0.0/24, 1 successors, FD is 2100                <--- Feasible Distance
        via 10.0.13.3 (2100/2050), GigabitEthernet0/3  <--- Successor Route
        via 10.0.12.2 (3000/2000), GigabitEthernet0/2  <--- Feasible Successor
                       |     |
                       |     +-- Reported Distance
                       +-------- Path Metric

                                                             (RD 2000 < FD 2100)

Unequal Cost Multi Path

EIGRP can load balance over the successor and feasible successor routes with a variance command.

Timers

• Hello packets are every 5 seconds, on 60 seconds on T1 links
  • The deadtime is 3x the hello timer

Initial Bringup

• Send Hello packets, to 224.0.0.10
  • Doesn’t’ require multicast to be on
  • Unicast Init from neighbor, set Seq, Set Ack to 0
    • Neighbor Sends back Ack as prior sequence number
    • Update Messages

Stuck in Active

• The router is too busy to answer the query (generally due to high CPU utilization)
• The router has memory problems and cannot allocate the memory to process the query or build the reply packet
• The circuit between the two routers is not good; there are not enough packets that get through to keep the neighbor relationship up, but some queries or replies are lost between the routers
• unidirectional links (a link on which traffic can only flow in one direction because of a failure)

Update Message

• AS number
• Prefixes
• End-of-table Flag

Prefixes

• Type (internal, etc)
• Reliability
• Load
• MTU
• Hop Count
• Delay
• Bandwidth
• Flags
  • Source Withdrawn
  • Candidate Default
  • Route is Active
  • Route is Replicated
• Next-hop
• Prefix Length

Auto Summary

Off by default on versions later than IOS 15.

The summarization done by this command is classful. This should never be turned on.

To enable:

auto-summary

Manual Summaries

In EIGRP these go under the interface, on the interface you want the summary to be sent out of.

ethernet 1
  ip summary-address eigrp 100 192.168.0.0/16

Named Mode

Name mode supports IPv6 inside a VRF.

Minimum Config

router eigrp EIGRP_100
 !
 address-family ipv4 unicast autonomous-system 100
  !
  network 0.0.0.0
  eigrp router-id 1.1.1.1
 exit-address-family

Using The Old Config, Then Having The Box Convert It For You

router eigrp 1
  eigrp upgrade-cli EIGRP_1

RIB Scaling

The Cisco RIB can only hold values that are unsigned 4 bytes. The EIGRP named metrics are 64-bit.

This is done automatically (and why the topology values don’t match “show ip route”. In the event you need to modify it, here it is.

router eigrp EIGRP_100
  address-family ipv4 unicast autonomous-system 100
    topology base
      metric rib-scale 100

Variance

Shorter Delays

In this example, the delay scale is 1x, 2x, 3x, 4x, 5x, 6x, 7x.

The lowest RIB FD is 433.

With a variance of two, only two interfaces get added to the RIB.

R1# show ip protocols | i eigrp|variance
Routing Protocol is "eigrp 100"
      Maximum metric variance 2

R1# show run | i int|delay
interface GigabitEthernet0/1
 delay 1
interface GigabitEthernet0/2
 delay 2
interface GigabitEthernet0/3
 delay 3
interface GigabitEthernet0/4
 delay 4
interface GigabitEthernet0/5
 delay 5
interface GigabitEthernet0/6
 delay 6
interface GigabitEthernet0/7
 delay 7

R1# show ip route

[output omitted]
!
! sorted to look pretty and be in order
!
D        2.2.2.2 [90/433] via 10.12.1.2, 00:02:35, GigabitEthernet0/1
                 [90/729] via 10.12.2.2, 00:02:35, GigabitEthernet0/2

Longer Delays

In this example, the delay scale is: 1x, 1.1x, 1.2x, 1.3x, 1.4x, 1.5x, 1.6x

The lowest FD is 3398.

With a variance of two, all seven interfaces get programmed.

R1# show ip protocols | i eigrp|variance
Routing Protocol is "eigrp 100"
      Maximum metric variance 2
!
! I configured delay, this is the correct way to alter metrics.
!
R1# show run | i int|delay
interface GigabitEthernet0/1
 delay 11
interface GigabitEthernet0/2
 delay 12
interface GigabitEthernet0/3
 delay 13
interface GigabitEthernet0/4
 delay 14
interface GigabitEthernet0/5
 delay 15
interface GigabitEthernet0/6
 delay 16
interface GigabitEthernet0/7
 delay 17

R1# show ip route

[output omitted]

!
! sorted to look pretty and be in order
!
D        2.2.2.2 [90/3398] via 10.12.1.2, 00:00:04, GigabitEthernet0/1
                 [90/3694] via 10.12.2.2, 00:00:04, GigabitEthernet0/2
                 [90/3991] via 10.12.3.2, 00:00:04, GigabitEthernet0/3
                 [90/4288] via 10.12.4.2, 00:00:04, GigabitEthernet0/4
                 [90/4584] via 10.12.5.2, 00:00:04, GigabitEthernet0/5
                 [90/4881] via 10.12.6.2, 00:00:04, GigabitEthernet0/6
                 [90/5177] via 10.12.7.2, 00:00:04, GigabitEthernet0/7

Network Parser

• The CLI parser is converting the IP into binary, then comparing it to the wild mask.
• The CLI parser will only save the matched bits of the IP.
• The CLI parser will not save the zeroth network, anything starting with 0.
• The CLI parser will only save the matched bits of an IP if if finds bits that are “on”
• Using the “all” mask of 255.255.255.255 creates this statement ‘network 0.0.0.0’ and matches everything.
• Using the “unique-ip” mask of 0.0.0.0 means “match this single address”
• The wildcard mask only accepts contiguous numbers “Discontiguous mask is not supported.”

192.0.2.5 127.255.255.255 - becomes 128.0.0.0, the rest of the bits get dropped.

References

Cisco - Understand and Use the Enhanced Interior Gateway Routing Protocol

Cisco - Configure EIGRP Named Mode

Cisco - Configuring EIGRP Wide Metrics

Cisco - How Does Unequal Cost Path Load Balancing (Variance) Work in IGRP and EIGRP

Cisco - Troubleshooting EIGRP Variance

Blame The Network - Stuck in Active

EIGRP Stub Routing

• Normally a hub-and-spoke technology.
• Deployed at the spoke sites.
• Simplifies EIGRP config.
• Prevents a stub site from being used as transit.
• Useful to limit the scope of a EIGRP query domain
  • Stub router replies to queries with inaccessible
    • Connected
    • Redistributed Static
    • External
    • Internal

References

IP Routing Configuration Guide, Cisco IOS XE 17.x - EIGRP Stub Routing Cisco IOS XE 17 - Cisco

EIGRP Classic Metric

The RFC recommended way to modify a path with EIGRP is changing the delay, under the interface. This will not impact other protocols. Modifying bandwidth … affects lots of things!

EIGRP classic mode 32-bit composite metric calculator per RFC 7868 section 5.6.1.1

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.1.1.  Classic Composite Formulation

   EIGRP calculates the composite metric with the following formula:

   metric = 256 * ({(K1*BW) + [(K2*BW)/(256-LOAD)] + (K3*DELAY)} *
            (K5/(REL+K4)))

   In this formula, Bandwidth (BW) is the lowest interface bandwidth
   along the path, and delay (DELAY) is the sum of all outbound
   interface delays along the path.  Load (LOAD) and reliability (REL)
   values are expressed percentages with a value of 1 to 255.

   Implementation note: Cisco IOS routers display reliability as a
   fraction of 255.  That is, 255/255 is 100% reliability or a perfectly
   stable link; a value of 229/255 represents a 90% reliable link.  Load
   is a value between 1 and 255.  A load of 255/255 indicates a
   completely saturated link.  A load of 127/255 represents a 50%
   saturated link.  These values are not dynamically measured; they are
   only measured at the time a link changes.

   Bandwidth is the inverse minimum bandwidth (in kbps) of the path in
   bits per second scaled by a factor of 10^7.  The formula for
   bandwidth is as follows:

                     (10^7)/BWmin

   Implementation note: When converting the real bandwidth to the
   composite bandwidth, truncate before applying the scaling factor.
   When converting the composite bandwidth to the real bandwidth, apply
   the scaling factor before the division and only then truncate.

   The delay is the sum of the outgoing interface delay (in tens of
   microseconds) to the destination.  A delay set to it maximum value
   (hexadecimal 0xFFFFFFFF) indicates that the network is unreachable.
   The formula for delay is as follows:

                     [sum of delays]

   The default composite metric, adjusted for scaling factors, for EIGRP
   is:

             metric = 256 * { [(10^7)/ BWmin] + [sum of delays]}

Validation

R1# show ip protocols | i weight
    Metric weight K1=2, K2=2, K3=2, K4=0, K5=0

R1# show ip eigrp topology 10.0.0.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(1.1.1.1) for 10.0.0.0/8
  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 6164
  Descriptor Blocks:
  192.168.12.2 (GigabitEthernet0/0), from 192.168.12.2, Send flag is 0x0
      Composite metric is (6164/5652), route is Internal
      Vector metric:
        Minimum bandwidth is 1000000 Kbit
        Total delay is 20 microseconds
        Reliability is 255/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 1
        Originating router is 2.2.2.2

References

RFC 7868 - Cisco’s Enhanced Interior Gateway Routing Protocol (EIGRP)

EIGRP Wide Metric

The RFC recommended way to modify a path with EIGRP is changing the delay, under the interface. This will not impact other protocols. Modifying bandwidth … affects lots of things!

This calculator uses the EIGRP values from the RFC, in table 5.6.1.2.

EIGRP named mode 64-bit wide metric calculator per RFC 7868 with correct interface delay constants

Wide Network Vectors

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.2.1.  Wide Metric Vectors

   EIGRP uses five "vector metrics": minimum Throughput, latency, load,
   reliability, and MTU.  These values are calculated from destination
   to source as follows:

              o Throughput    - Minimum value
              o Latency       - accumulative
              o Load          - maximum
              o Reliability   - minimum
              o MTU           - minimum
              o Hop count     - accumulative


   There are two additional values: Jitter and energy.  These two values
   are accumulated from destination to source:

           o Jitter - accumulative
           o Energy - accumulative

Wide Metric Conversion Constants

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.2.2.  Wide Metric Conversion Constants

   EIGRP uses a number of defined constants for conversion and
   calculation of metric values.  These numbers are provided here for
   reference

           EIGRP_BANDWIDTH                    10,000,000
           EIGRP_DELAY_PICO                    1,000,000
           EIGRP_INACCESSIBLE       0xFFFFFFFFFFFFFFFFLL
           EIGRP_MAX_HOPS                            100
           EIGRP_CLASSIC_SCALE                       256
           EIGRP_WIDE_SCALE                        65536

   When computing the metric using the above units, all capacity
   information will be normalized to kilobytes and picoseconds before
   being used.  For example, delay is expressed in microseconds per
   kilobyte, and would be converted to kilobytes per second; likewise,
   energy would be expressed in power per kilobytes per second of usage.

Throughput

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.2.3.  Throughput Calculation

   The formula for the conversion for Max-Throughput value directly from
   the interface without consideration of congestion-based effects is as
   follows:

                                  (EIGRP_BANDWIDTH * EIGRP_WIDE_SCALE)
        Max-Throughput = K1 *     ------------------------------------
                                       Interface Bandwidth (kbps)

   If K2 is used, the effect of congestion as a measure of load reported
   by the interface will be used to simulate the "available Throughput"
   by adjusting the maximum Throughput according to the formula:

                                           K2 * Max-Throughput
        Net-Throughput = Max-Throughput + ---------------------
                                              256 - Load

   K2 has the greatest effect on the metric occurs when the load
   increases beyond 90%.

Latency

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.2.4.  Latency Calculation

   Transmission times derived from physical interfaces MUST be n units
   of picoseconds, converted to picoseconds prior to being exchanged
   between neighbors, or used in the composite metric determination.

   This includes delay values present in configuration-based commands
   (i.e., interface delay, redistribute, default-metric, route-map,
   etc.).

   The delay value is then converted to a "latency" using the formula:

                          Delay * EIGRP_WIDE_SCALE
        Latency = K3 *   --------------------------
                             EIGRP_DELAY_PICO

Composite Calculation

RFC 7868                      Cisco's EIGRP                     May 2016

5.6.2.5.  Composite Calculation

                                                                K5
      metric =[(K1*Net-Throughput) + Latency)+(K6*ExtAttr)] * ------
                                                              K4+Rel

   By default, the path selection scheme used by EIGRP is a combination
   of Throughput and Latency where the selection is a product of total
   latency and minimum Throughput of all links along the path:

      metric = (K1 * min(Throughput)) + (K3 * sum(Latency)) }

Validations

R1# show eigrp address-family ipv4 topology 2.2.2.2/32
EIGRP-IPv4 VR(EIGRP_100) Topology Entry for AS(100)/ID(1.1.1.1) for 2.2.2.2/32
  State is Passive, Query origin flag is 1, 7 Successor(s), FD is 1392640, RIB is 10880
  Descriptor Blocks:
  10.12.1.2 (GigabitEthernet0/1), from 10.12.1.2, Send flag is 0x0
      Composite metric is (1392640/163840), route is Internal
      Vector metric:
        Minimum bandwidth is 1000000 Kbit
        Total delay is 11250000 picoseconds
        Reliability is 255/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 1
        Originating router is 2.2.2.2

Validation

R1# show ip protocols | i weight
    Metric weight K1=1, K2=2, K3=3, K4=4, K5=5 K6=0


R1# show ip eigrp topology 2.2.2.2/32
EIGRP-IPv4 VR(EIGRP_100) Topology Entry for AS(100)/ID(1.1.1.1) for 2.2.2.2/32
  State is Passive, Query origin flag is 1, 7 Successor(s), FD is 55450, RIB is 433
  Descriptor Blocks:
  10.12.1.2 (GigabitEthernet0/1), from 10.12.1.2, Send flag is 0x0
      Composite metric is (55450/6338), route is Internal
      Vector metric:
        Minimum bandwidth is 1000000 Kbit
        Total delay is 11250000 picoseconds
        Reliability is 255/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 1
        Originating router is 2.2.2.2

References

RFC 7868 - Cisco’s Enhanced Interior Gateway Routing Protocol (EIGRP)

IS-IS

What is IS-IS

• Intermediate System To Intermediate System
• An ISO standard open protocol.
• Link State and Shortest Path
• Good for large flat networks

Terms

IS

• Intermediate System
• A router

NSAP

• Network Service Access Point

NET

• Network Entity Title
• A router
• Also refers to the address NSAP

ES

• End Station
• A PC, or a server.

Station Routing

• AKA, intra-area
• Routing within a L1 area.

Area Routing

• AKA, inter-area
• Routing within a L2 area.
• The L2 area.

Types of Routers

L2 Routers

IS-IS doesn’t refer to a backbone, but L2 routers perform the same function. They should be center-of-topology.

L1L2 Routers

These routes have topology information for the L1 area and the L2 area.

These are kind of like ABRs in OSPF.

L1 Router

These are the Area routers. They do not flood their link state databases into L2.

• Intra-area
• Default route out (sets the attached bit)
• Redistribution is allowed

Example

                                             ┌──────┐
                                             │ L1   │
                                             └───┬──┘
                                                 │
┌──────┐     ┌──────┐        ┌──────┐        ┌───┴──┐     ┌─────┐
│  L1  ├─────┤ L1L2 ├────────┤  L2  ├────────┤ L1L2 ├─────┤ L1  │
└───┬──┘     └───┬──┘        └──────┘        └───┬──┘     └─────┘
    │            └──────┐                  ┌─────┘
┌───┴──┐             ┌──┴───┐          ┌───┴──┐
│  L1  │             │  L2  │──────────┤  L2  │
└──────┘             └──────┘          └──────┘

Topologies

Single Topology

• All Routed Protocols must be configured on all enabled interfaces.
• e.g. v4 and v6 on all interfaces.

Multi-Topology

• Some interfaces can be v4, others can be v6, others can be both.

Addressing scheme

packet-beta
0-7: "AFI"
8-23: "Area ID"
24-71: "System ID"
72-79: "SEL"

AFI

• Authority and Format Identifier - 1 byte
• 49 means local authority, and hexadecimal (binary is encoded).

Area ID

• Variable, and … includes the AFI.

System ID

• 6 bytes, can fit a MAC address or a v4 address.
• Must be unique in an area for L1.
• Must be unique in a domain for L2.

SEL

• Selector - 1 byte.
• This is always 00 to mean router.

Example

net 49.0001.0000.0A00.0001.00

So long as the NSAP is unique, its OK because we aren’t routing CLNS.

Priority is used for the CLNS election. Circuit ID, who won the election

Etc

ISIS does not ride IP, it rides CLNS. To do Multipoint NBMA you need to include CLNS resolution.

L1 areas must match

IS-IS Narrow

The Cisco default link cost is 10.

These are the limits:

• 63 per link
• 1 023 per path

IS-IS Wide

• 16 777 215 per link
• 4 294 967 295 per path

Cisco’s implementation of the wide metric uses the bits ISO set aside for delay, expense and error.

Config

Enable wide metrics:

metric-style wide

Metric Transition Commands

Used when migrating from narrow to wide without a hard cutover:

IS-IS Authentication

• Plaintext
  • Link, Area, or Domain
    • Link is between routers
    • Area is every router must have a matching password
    • L2 and L1/L2 router use domain authentication.

Notes

Default route injected via route-map.

References

RFC 1195: Use of OSI IS-IS for routing in TCP/IP and dual environments | RFC Editor

RFC 5308: Routing IPv6 with IS-IS | RFC Editor

ISO/IEC 10589:2002 - Information technology — Telecommunications and information exchange between systems

ISO/IEC 8348:2002 - Information technology — Open Systems Interconnection — Network service definition

IS-IS Network Design

Flat

Small Networks

Hybrid

Mid Sized Networks, still allows some summarization.

Hierarchical

Large Networks, can be designed to support summarization.

References

A. Bruno and S. Jordan, CCNP Enterprise Design ENSLD 300-420 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2024.

IPv4 to Hex

This uses in browser javascript.

IPv4 to hexadecimal converter

BGP

ASN

16-bit ASN - 65,535 32-bit ASN - 4,294,967,295

Private Numbers

• 64,512 – 65,534
• 4,200,000,000 – 4,294,967,294

BGP Path Attributes

RFC 4271 - BGP-4

• Well-known mandatory
• Well-known discretionary
• Optional transitive
• Optional nontransitive

BGP Uses TCP

• Port 179
• BGP is sensitive to IP fragmentation

Session Types

• iBGP Administrative distance of 200
• eBGP Administrative distance of 20

eBGP

• TTL is set to 1.
• Next-hop is set to what the BGP source connection IP is.
• Check if the current AS_PATH has our AS.
• Prepend AS into AS_PATH

BGP Packet Types

Theory

• BGP works on the premise that if a router sees its own AS path, it must be a loop.
• The default timer is 60 seconds with 180 seconds for hold time. This means worst-case is 3 minutes to fail-over.
• BGP aggregate-address only works if there is a subnet inside the aggregate range in BGP.

Working With BGP

• Only consider traffic in one direction at a time
• Accepting a route will affect outgoing traffic
• Advertising a route will affect incoming traffic
• Filter out everything except the routes needed
• BGP DOES NOT LOAD BALANCE

On Cisco IOS bgp soft-reconfig-backup tells the router “if you must, save a entire table” otherwise rely on RFC2918, which are dynamic updates.

Soft reconfig is ancient, pre-RFC.

Soft Reconfig via Route Refresh (trusting the other device)

clear ip bgp <neighbor_ip> soft in¹

BGP Best Path Selection

- Higher Weigth                                       
- Higher Local Preference                            
- Locally Originated                                 (Network or Aggregate Command)
- Shortest AS-PATH
- Lowest Origin Type                                 (IGP > EGP > Incomplete)
- Lowest MED                                         (Neighbor ASes must be the same)
- Prefer eBGP > Confederated eBGP > iBGP
- Prefer path with lowest IGP metric to next hop
- Determine if bestpath is enabled
  - Prefer external path which is oldest
  - Prefer path from router with lower ID
  - Prefer path with shorter cluster length
  - Prefer path from lowest neighbor address

Cisco - Select BGP Best Path Algorithm

WEIGHT

• Cisco specific & this router only
• Routes learned are 0
• Locally generated routes are 32768

LOCAL_PREF

• Controls traffic Outgoing traffic.
• Only shared between iBGP peers, used to determine the exit. Higher is better.

AS Path

These read left to right like a book. This prefix was most recently from AS 7018.

7018 701 15 i
            ^ this means IGP, and AS 15 has an IGP route for it like OSPF or EIGRP

Next Hop

1. eBGP, routers in different AS, destination outside AS. The Next hop will be the advertising router.
2. iBGP, routers in same AS, destination inside AS. The Next hop will be the advertising router.
3. iBGP, routers in same AS, destination outside AS. The Next hop is the external peer who advertised the address.

… When the third option happens …

• Advertise into the IGP the external links to the BGP peers.
• Tell the AS border router to change the next hop to its own IP address. (next-hop-self)

Origin

IGP > EGP > Incomplete

• IGP means it came from an IGP. This is the highest preference.
• Incomplete means its likely a redistributed route

MULTI_EXIT_DISC

• Controls incoming traffic.
• Lower is better

ATOMIC_AGGREGATE

BGP can aggregate smaller prefixes into larger ones even if a smaller prefix comes from a different AS.

A router in AS 105 gets these prefixes from its peers.

192.168.0.0/24 (123 204)
192.168.1.0/24 (123 205)

If the administrator chooses, they can aggregate this, but lose path information.

192.168.0.0/23 (105) ATOMIC_AGGREGATE. 

Downstream peers can not remove this tag

AGGREGATOR

AS and Router ID of the BGP router that did the atomic aggregation.

COMMUNITY

Usually used to tag routes from a specific customer.

ORIGINATOR_ID

For route reflectors The origaning router puts its Router_ID here. If it sees this, it knows a loop as occured.

BGP By Default Will Summarize

Use no auto-summary.

Using redistribute under BGP will make the resulting route show up with an orign code of incomplete.

Sending A Default Route

neighbor A.B.C.D default-originate

To get iBGP routers to update the next-hop to be themselves when advertising to other iBGP routers use

neighbor A.B.C.D next-hop-self

This makes it so other iBGP routers don’t need reachability information for the physical link to the next AS.

BGP Neighbor States

Idle

Connect

Open Sent

Open Confirm

Active

Established

Fixing Next-Hop Issues

Just because the route shows up in show ip bgp doesn’t mean it will install. BGP needs to be able to reach the next-hop.

1. Add the transit routes the IGP.
2. Use next-hop self in BGP.
3. Use a route-map to set the next hops.

References

RFC 4271: A Border Gateway Protocol 4 (BGP-4) | RFC Editor

The Network Times: Border Gateway Protocol – Finite State Machine (BGP-FSM)

BGP Troubleshooting Diagram

V. Jain and B. Edgeworth, Troubleshooting BGP: A Practical Guide to Understanding and Troubleshooting BGP, 1st ed. Indianapolis, IN: Cisco Press, 2016, ISBN 978-1-58714-464-6.

B. Edgeworth, R. Garza Rios, J. Gooley, and D. Hucaby, CCNP and CCIE Enterprise Core ENCOR 350-401 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

I. van Beijnum, Internet Routing with BGP, Kindle ed., Nov. 12, 2022, 269 pp.

1. https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-16/irg-xe-16-book/bgp-4-soft-configuration.html ↩

BGP Neighbor FSM

Events

There are 28 events total.

Administrative Events

Things the operator can set.

• Event 1: ManualStart
• Event 2: ManualStop
• Event 3: AutomaticStart
• Event 4: ManualStart_with_PassiveTcpEstablishment
• Event 5: AutomaticStart_with_PassiveTcpEstablishment
• Event 6: AutomaticStart_with_DampPeerOscillations
• Event 7: AutomaticStart_with_DampPeerOscillations_and_PassiveTcpEstablishment
• Event 8: AutomaticStop

Timer Events

• Event 9: ConnectRetryTimer_Expires
• Event 10 HoldTimer_Expires
• Event 11 KeepaliveTimer_Expires
• Event 12 DelayOpenTimer_Expires
• Event 13 IdleHoldTimer_Expires

TCP connection based events

Packet based, not necessarily the BGP app, but the underlying transport.

• Event 14: TcpConnection_Valid
• Event 15: Tcp_CR_Invalid
• Event 16: Tcp_CR_Acked
• Event 17: TcpConnectionConfirmed
• Event 18: TcpConnectionFails

BGP message based events

Sent or received over TCP.

• Event 19: BGPOpen
• Event 20: BGPOpen with DelayOpenTimer running
• Event 21: BGPHeaderErr
• Event 22: BGPOpenMsgErr
• Event 23: OpenCollisionDump
• Event 24: NotifMsgVerErr
• Event 25: NotifMsg
• Event 26: KeepAliveMsg
• Event 27: UpdateMsg
• Event 28: UpdateMsgErr

BGP Confederation

NEXT_HOP is preserved throughout the confederation.

MED is preserved for routes advertised into the confederation.

LOCAL_PREF is preserved throughout the confederation.

AS_PATH for privates ASes is used within the confederation.

Force Interior Confederation MEDs To Be Considered

bgp deterministic-med

Route Reflectors are generally preferred.

IF you want to add two BGP speakers to the same router reflector cluster, specify the cluster ID.

• clients can not detect inter-cluster loops. They don’t have the attributes in the BGP table.

BGP Route Reflector

RR — Route Reflector

Cluster

A route reflector and its clients.

Cluster List

The sequence of Cluster_IDs through which the route has passed. If a router sees its own Router_ID a loop has occurred.

Originator_ID

• The router that introduced the route into the AS.
• Used to prevent loops between clients.

Route Reflection Rules

A RR will not change any attributes of a route.

1. If a route is learned from a non-client iBGP peer, reflect to clients
2. If a route is learned from a client, reflect to everyone
3. If a route is learned from a eBGP peer, reflect to everyone

Notes

• Route reflectors can be clients of each other. This causes extra overhead.
• If multiple route reflectors server the same cluster they should have the same Cluster_ID.

BGP Route Reflectors Loop Prevention

• If a BGP router that receives a route from an iBGP neighbor in the incoming update detects the presence of its own Router-ID in the Originator-ID attribute it will reject the update.
• If a BGP router that receives a route from an iBGP neighbor is configured to operate as a route reflector and in the incoming update detects the presence of its own Cluster-ID in the Cluster-list attribute it will reject the update.

Only the route reflector is aware of the reflecting. The clients are dumb

If you configure route reflectors as a cluster you must manually configure the cluster_ID

BGP Multipath

BGP on its own will not install multiple paths like OSPF or EIGRP to reach a destination.

To make BGP behave more like an IGP, and especially at scale with multiple ECMP links, this feature is needed.

References

BGP Multipath and Load Balancing Techniques

IP Routing: BGP Configuration Guide - iBGP Multipath Load Sharing Cisco ASR 1000 Series Aggregation Services Routers - Cisco

BGP Tuning

1. Enabling TCP Path MTU discovery for every neighbor, to allow the TCP selecting optimum MSS size. Notice that this requires that no firewall blocks the ICMP unreachable messages used during the

   discovery process.

2. Tuning the router’s ingress queue size to allow for successful absorption of large amount of TCP ACK messages. When a router starts replicating BGP UPDATES to its peers, every peer responds with TCP ACK message to normally every second segment sent (TCP Delayed ACK). The more peers router has, the higher will be the pressure on the ingress queue.

References

Verify Path MTU Discovery on Cisco IOS XR and BGP - Cisco

BGP load Balancing

Inbound

• See if the ISP supports BGP Communities with local preference to mark routes on their side
• Adjust the MED, lower is better.
• Advertise longer prefixes to both ISPs
• Use AS_Path prepending

Outbound

• Use Weight
• Use Local Preference
• Advertise a default route into the network
• Filter the provider routes and only install a subset into the RIB.
• Use AS_Path filtering

BGP Multipath

• Uses the maximum-paths keyword.

Do not become a Transit Network

• Filter your routes, only advertise subnets you own.

Route Redistribution

Terms

Administrative Distance

How trusted a route is, if two routes have the same subnet and mask.

One-way Redistribution

Useful for WAN edges, to learn a few subnets and add them into the core. The WAN network or branch site gets a default route.

Two-way Redistribution

Used nearer the core, the router redistributes both ways. This can cause feedback loops without techniques like route filters.

Seed Metric

The values given to the receiving routing protocol.

Route Loop

Traffic bounces back-and-forth between two routers until the TTL reaches 0.

Route Feedback

A redistributed route in a protocol with a better AD, is fed back into a protocol with a lower AD.

Route feedback can be fixed by modifying Administrative Distance with distance commands.

Rules

Non-Transitive

If a router is running three routing protocols, for full-reachability, each routing protocol needs the other two protocols redistributed into each other.

Routes Must Be In the RIB

The router doing the redistribution needs to have the route installed.

Seed Metrics Matter

Fast links should get better seed metrics that slower links.

Always provide a meaningful seed metric.

Troubleshooting

debug ip routing

Show changes to the routing table.

Seed Metric Preference

If multiple seed metrics are applied, this is the order of preference.

1. Route Map applied redistribute
2. redistribute
3. default-metric

References

B. Edgeworth and R. Lacoste, CCNP Enterprise Advanced Routing ENARSI 300-410 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

Route Redistribution BGP

BGP defaults allow only eBGP learned routes to be redistributed.

To change this use bgp redistribute-internal.

Caution

Redistribution of iBGP routes into an IGP can cause routing loops.

The IGPs should have route filters set.

Seed Metric

Redistribute Transport Networks

This is the safe way to redistribute connected networks. Useful for transit or peering links.

router bgp 65100
 address-family ipv4
  redistribute connected route-map RM_BGP_LOOPBACK
!
route-map RM_BGP_LOOPBACK permit 10
 match interface Loopback0

Redistribute All OSPF Routes

router bgp 100
 redistribute ospf 1 match internal external 1 external 2 

References

Understand the Redistribution of OSPF Routes into BGP - Cisco

B. Edgeworth and R. Lacoste, CCNP Enterprise Advanced Routing ENARSI 300-410 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

Route Redistribution EIGRP

Seed Metric

Set Seed Metric

default-metric bandwidth delay reliability load mtu

Route Feedback Solutions

The Problem

A route comes from RIP and moves through the routing domain like this:

RIP -> EIGRP -> OSPF -> EIGRP
120 -> 170   -> 110  -> 170

The Solution

Set EIGRP external to be a lower AD than OSPF.

router eigrp EIGRP_100
 !
 address-family ipv4 unicast autonomous-system 100
  !
  topology base
   distance eigrp 90 100
  exit-af-topology
 exit-address-family

References

Configure Mutual Redistribution Between EIGRP and BGP - Cisco

B. Edgeworth and R. Lacoste, CCNP Enterprise Advanced Routing ENARSI 300-410 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

Route Redistribution OSPF

Seed Metrics

Redistribution

The subnets keyword is required, or OSPF will work in a classful way.

redistribute source-protocol [subnets] [metric metric] [metric-type {1 | 2}] [tag 0-4294967295] [route-map route-map-name]

Distance

Modify the AD for different kinds of LSAs.

distance ospf {external | inter-area | intra-area} ad

Modify the AD for specific routes.

distance ad source-ip source-ip-wildcard [acl-number | acl-name]

References

Redistribute OSPF Among Different OSPF Processes - Cisco

B. Edgeworth and R. Lacoste, CCNP Enterprise Advanced Routing ENARSI 300-410 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

Route Redistribution with Tags

• Tag EIGRP as 100
• TAG OSPF as 1
• Route maps should take the form DENY -> PERMIT.
• Routes are tagged when they are advertised.

Route tags appear on-the-wire and can be read by other routers.

ospf.lsa.asext.extrttag == 100

In this example, EIGRP becomes a Type-5 OSPF update, with a route-tag of 100. If we look for these tags can exclude them in redistribution updates.

route-map RM_OSPF_INTO_EIGRP deny 10
 description previously tagged EIGRP traffic
 match tag 100
!
route-map RM_OSPF_INTO_EIGRP permit 20
 match source-protocol ospf 1 
 set tag 1
!
route-map RM_EIGRP_INTO_OSPF deny 10
 description previously tagged OSPF traffic
 match tag 1
!
route-map RM_EIGRP_INTO_OSPF permit 20
 match source-protocol eigrp 100
 set tag 100
!
router eigrp 100
 redistribute ospf 1 metric 1000000 100 255 1 1500 route-map RM_OSPF_INTO_EIGRP
!
router ospf 1
 redistribute eigrp 100 subnets route-map RM_EIGRP_INTO_OSPF

References

Configure Routing Protocol Redistribution - Cisco

Policy Based Routing

The more advanced of a match operation, the harder this becomes to get correct.

Verify with Flexible Netflow.

Config

A host is doing a speedtest, with iperf. I want to match that traffic, and route it a longer way.

ip access-list extended AL_IPERF_TO_IPERF
  10 permit udp host 10.0.100.100 eq 2000 host 10.0.200.200
!
route-map RM_IPERF_TO_IPERF permit 10 
  match ip address AL_IPERF_TO_IPERF
  set ip next-hop 10.2.3.3
!
interface GigabitEthernet1
  ip policy route-map RM_IPERF_TO_IPERF

Verify

• Destination Routing says Take Gig3
• PBR says take Gig2

R2# show ip route 10.0.200.200
Routing entry for 10.0.200.0/24
  Known via "ospf 1", distance 110, metric 2, type intra area
  Last update from 10.2.4.4 on GigabitEthernet3, 01:03:09 ago
  Routing Descriptor Blocks:
  * 10.2.4.4, from 4.4.4.4, 01:03:09 ago, via GigabitEthernet3
      Route metric is 2, traffic share count is 1

R2# show flow monitor FLOW_MONITOR_IPV4 cache sort highest counter bytes long top 10 format table
Processed 2 flows
Aggregated to 2 flows
Showing the top 2 flows

IPV4 SRC ADDR    IPV4 DST ADDR    TRNS SRC PORT  TRNS DST PORT  INTF INPUT            IP PROT  intf output                     bytes long             pkts long    time first     time last
===============  ===============  =============  =============  ====================  =======  ====================  ====================  ====================  ============  ============
10.0.100.100     10.0.200.200              2000          42970  Gi1                        17  Gi2                                 172704                   168  01:39:42.020  01:40:15.368
10.1.2.1         224.0.0.5                    0              0  Gi1                        89  Null                                   100                     1  01:40:06.846  01:40:06.846

References

Cisco - Policy Based Routing - IOS-XE 17.x

Cisco - Best Practices for Route Maps

Cisco - Configure Policy-based Routing with Next-Hop Commands

Encryption Fundamentals

Terms

One-Way Encryption

• Bob encrypts cleartext for Alice using her public key
• Afterwards Bob cannot decrypt the ciphertext

Two-Way Encryption

• Bob encrypts cleartext for Alice
• They can both decrypt the ciphertext
  • Uses a shared symmetric key

Symmetric Encryption

• A two-way encryption key
• Very fast to use

Asymmetric Encryption

Slow encryption, with two parts:

• Public Key - used for one way encryption.
• Private Key - used for one way decryption.

DH — Diffie Hellman

• 1976 method of exchanging secret symmetric keys using asymmetric keys

References

Diffie–Hellman key exchange - Wikipedia

IKE

IKE uses UDP port 500.

All of IKE is Request Response Pairs.

Terms

IKE – Internet Key Exchange

SA — Security Association

• Shared Secret
• Set of Agreed on and Shared Cryptographic algorithms to transport information

Message ID

• Requests and Responses share the same Message ID
• 32 bits

Initiator

• Proposes a cryptographic suite

Responder

• Accepts or denies the requests

ISAKMP – Internet Security Association and Key Management Protocol

• One method to perform key exchange.

venn-beta
  set A["IKE Key Exchanges"]
    text A1["OAKLEY"]
    text A2["SKEME"]
  set B["Cisco Implemented"]
  union A,B
    text AB["ISAKMP"]

Requirements

IKE cannot be fragmented beyond 1280.

Retransmissions use the same Message ID.

Responses use the same Message ID.

Process Flow

sequenceDiagram
    participant I as IKEv2 Initiator
    participant R as IKEv2 Responder

    rect rgb(240, 248, 250)
    note over I,R: Initial Exchanges
    I->>R: IKE_SA_INIT Request
    R-->>I: IKE_SA_INIT Response
    I->>R: IKE_AUTH Request
    R-->>I: IKE_AUTH Response
    end

    rect rgb(252, 244, 240)
    note over I,R: CREATE_CHILD_SA Exchange
    I->>R: CREATE_CHILD_SA Request
    R-->>I: CREATE_CHILD_SA Response
    end

IKE_SA_INIT

• Negotiate Cryptographic Algorithms
• Nonce exchange
• DH exchange

IKE_AUTH

• Encrypted using IKE_SA_INIT
  • Authenticates Previous Messages
  • Exchange Identities and certificates
  • Establish first child SA

CREATE_CHILD_SA

Used for dataplane traffic.

References

What Is IKE (Internet Key Exchange)? | IKE Meaning - Palo Alto Networks

Understand IPsec IKEv1 Protocol - Cisco

RFC 7296: Internet Key Exchange Protocol Version 2 (IKEv2) | RFC Editor

Cisco Secure VPNS

IPSec

IPSec with IKEv2 in Tunnel mode is an Industry Standard.

• OK for point-to-point
• No Dynamic Routing
• No Multicast

GRE IPSec

• Tunnels Multicast

GETVPN

• Cisco Proprietary
• One SA for the entire group
• Native Multicast

DMVPN

• Automates the setup of point-to-point SAs.
• Works over the Internet
• Tunnels Multicast

SD-WAN

• Controller based
• Supports ZTP
• Managed via webpage
• Tunnels Multicast

References

Cisco VPN Technologies Reference Guide - Cisco

Cisco Site-to-Site VPN Technologies Comparison - 2006

GRE

GRE — Generic Routing Encapsulation

• No encryption
• Just a transport service

To carry IPv4 just set the protocol type to 0x800

The outer header (the delivery header) if it’s IPv4 uses the protocol 47

Structure

GRE is a generic transport, it has three parts:

┌───────────────────────────────┐ 
│                               │ 
│       Delivery Header         │ 
│                               │ 
├───────────────────────────────┤ 
│                               │ 
│       GRE Header              │ 
│                               │ 
├───────────────────────────────┤ 
│                               │ 
│       Payload packet          │ 
│                               │ 
└───────────────────────────────┘ 

GRE Header

This is a RFC 2784 GRE header.

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |C|       Reserved0       | Ver |         Protocol Type         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |      Checksum (optional)      |       Reserved1 (Optional)    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

References

RFC 2784: Generic Routing Encapsulation (GRE) | RFC Editor

IPsec

Terms

DARPA — Defense Advanced Research Projects Agency, USA.

In the 1970s DARPA was tasked with solving packet encryption.

NSA – National Security Agency, USA.

In 1986 NSA sponsored security protocols under the Secure Data Network Systems (SDNS) program.

NRL — Naval Research Laboratory, USA.

In 1992, NRL was funded by DARPA to implement IPv6 and research packet encryption in BSD 4.4.

History

DARPA makes its packet encryption implementation available under a MIT license, the NRL starts work to push it into an IETF standard.

The NRLs version is also released as MIT and becomes the basis for most commercial implementations, RFCs 1825, 1826, and 1827.

Purpose

Unlike SSH, or TLS which work at upper layers, IPSec can secure the packets themselves.

References

IPsec - Wikipedia

RFC 4301: Security Architecture for the Internet Protocol | RFC Editor

IPSec Authentication Header

IP Protocol 51.

Protections

• No options insertion
• No altering the IP payload

References

IPsec - Wikipedia

IPSec Encapsulating Security Payload

IP Protocol 50

Usually used with GRE, or mGRE.

Terms

ESP — Encapsulating Security Payload

• IPSec process responsible for providing encryption

ESP SPI — Security Parameters Index

• 32-bit field
• Identifies the SA on both sides

ESP Sequence

• 32-bit field
• Goes up by 1 for each transmitted packet.

It’s not recommended to share a SA for multiple senders for this reason.

Transport Mode

Tunnel Mode

References

RFC 4303: IP Encapsulating Security Payload (ESP) | RFC Editor

RFC 7296: Internet Key Exchange Protocol Version 2 (IKEv2) | RFC Editor

IPsec - Wikipedia

DMVPN

DMVPN has a lot of benefits:

• One GRE interface
• (optional) One IPSec Profile for all spoke routers
  • IPsec transport
  • Automatic and Dynamic IPSec initiation
• Remote spokes can by dynamically addressed
• Remote spokes can be behind NAT
  • Hub can be behind static NAT
• Supports Dynamic spoke-to-spoke with phases 2 and 3
• QoS
• Dynamic Routing
• OKish Multicast
  • IOS-XE Supports PIM-SM

Terms

DMVPN — Dynamic Multipoint VPN

Underlay

• Sometimes dynamically addressed
• AKA NBMBA
• AKA The ISP Network

Overlay

• Usually one /24
• Used on the mGRE tunnel interfaces

Caveats

Warning

OSPF point-to-multipoint needs the following config

distribute-list prefix-list PL_BLOCK_OSPF_32 out   
  ip prefix-list PL_BLOCK_OSPF_32 deny <tunnel-subnet> <mask> ge 32 
  ip prefix-list PL_BLOCK_OSPF_32 permit any le 32

This is why.

This is a Cisco documented failing

Phase 1

• GRE
• Only hub-to-spoke

       ┌─────────┐
       │  hub 1  │
       └┬──────┬─┘
     ┌──┘      └──┐
┌────┴────┐   ┌───┴─────┐
│ spoke 1 │   │ spoke 2 │
└─────────┘   └─────────┘

Phase 2

• mGRE
• Spoke-to-spoke
• Supports Hierarchical Tree Topology, multiple hubs

                           ┌─────────┐
                           │  hub 2  │
                           └─┬─────┬─┘
              ┌──────────────┘     └────────────────┐
       ┌──────┴──┐     ┌───────────────────┐     ┌──┴──────┐
       │  hub 1  │     │     Phase 2       │     │  hub 3  │
       └┬──────┬─┘     │                   │     └─┬──────┬┘
     ┌──┘      └─────┐ ▼                   ▼  ┌────┘      └──┐
┌────┴────┐   ┌──────┴──┐               ┌─────┴───┐    ┌─────┴───┐
│ spoke 1 │   │ spoke 2 │               │ spoke 3 │    │ spoke 4 │
└─────────┘   └─────────┘               └─────────┘    └─────────┘

Phase 3

• NHRP Path Summarization
• NHRP Shortcuts
• NHRP Redirects
• Hierarchical Tree Topology, multiple hubs, with redirects and shortcuts.

                           ┌─────────┐
                           │  hub 2  │
                           └─┬─────┬─┘
              ┌──────────────┘     └────────────────┐
       ┌──────┴──┐                               ┌──┴──────┐
       │  hub 1  │                               │  hub 3  │
       └┬──────┬─┘                               └─┬──────┬┘
     ┌──┘      └─────┐                        ┌────┘      └──┐
┌────┴────┐   ┌──────┴──┐    Phase 3    ┌─────┴───┐    ┌─────┴───┐
│ spoke 1 │   │ spoke 2 │ ◄───────────► │ spoke 3 │    │ spoke 4 │
└─────────┘   └─────────┘               └─────────┘    └─────────┘

Config

Hub

interface Tunnel1
 ip address 192.168.100.1 255.255.255.0
 ip nhrp network-id 111
 ip nhrp redirect
 !
 ! This is the NBMA address.
 !
 tunnel source 10.0.110.1
 tunnel mode gre multipoint

Spoke

interface Tunnel1
 ip address 192.168.100.2 255.255.255.0
 no ip redirects
 !
 ! Logical address, then NBMA address
 !
 ip nhrp shortcut
 ip nhrp map 192.168.100.1 10.0.110.1
 ip nhrp map multicast 10.0.110.1
 ip nhrp network-id 111
 ip nhrp nhs 192.168.100.1
 tunnel source 10.0.120.2
 tunnel mode gre multipoint

References

Cisco - Technote - Configure BGP over DMVPN Phase 3

Cisco - Datasheet - Dynamic Multipoint VPN: Simple and Secure Branch-to-Branch Communications

Cisco - Concepts & Configuration - DMVPN

Cisco - Security and VPN Configuration Guide, Dynamic Multipoint VPN - IOS-XE 17

Cisco - IP Multicast Configuration Guide, Dynamic Multipoint VPN - IOS-XE 17

Wikipedia - Dynamic Multipoint Virtual Private Network

Send The Payload - Just A Bunch of DMVPN Configuration Examples

Yasser Auda - CCIEv5 DMVPN Lab Workbook

This Bridge is the Root - DMVPN Deep Dive – NHRP, mGRE, Routing Scenarios and IPsec

B. Edgeworth and R. Lacoste, CCNP Enterprise Advanced Routing ENARSI 300-410 Official Cert Guide, 2nd ed. Indianapolis, IN: Cisco Press, 2023.

NHRP

NHRP — Next-Hop Resolution Protocol

NBMA — Non-Broadcast Multi-Access Network

• Network connections you could make
• If you knew how to address the packet

NBMA Address

• AKA transport address
• DMVPN, public Internet address
• Frame-relay, DLCI

NHS — Next Hop Server

• Typically the hub router
• Supports dynamic registration

NHC — Next Hop Client

• Dynamically register with the NHS

Protocol Address

• AKA overlay address
• Usually a /24

Config

Hub

interface Tunnel1
 ip address 192.168.100.1 255.255.255.0
 no ip redirects
 ip nhrp network-id 111
 !
 ! This is the NBMA address.
 !
 tunnel source 10.0.110.1
 tunnel mode gre multipoint

Spoke

interface Tunnel1
 ip address 192.168.100.2 255.255.255.0
 no ip redirects
 !
 ! Logical address, then NBMA address
 !
 ip nhrp map 192.168.100.1 10.0.110.1
 ip nhrp map multicast 10.0.110.1
 ip nhrp network-id 111
 ip nhrp nhs 192.168.100.1
 tunnel source 10.0.120.2
 tunnel mode gre multipoint

Verification

This hub knows about two sites, that have dynamically registered their NBMA addresses.

hub# show ip nhrp brief

****************************************************************************
    NOTE: Link-Local, No-socket and Incomplete entries are not displayed
****************************************************************************
Legend: Type --> S - Static, D - Dynamic
        Flags --> u - unique, r - registered, e - temporary, c - claimed
        a - authoritative, t - route
============================================================================

Intf     NextHop Address                                    NBMA Address
         Target Network                              T/Flag
-------- ------------------------------------------- ------ ----------------
Tu1      192.168.100.2                                      10.0.120.2
         192.168.100.2/32                            D/r

Tu1      192.168.100.3                                      10.0.130.3
         192.168.100.3/32                            D/r

References

Cisco - IOS IP Routing: NHRP Command Reference - IOS XE 16

RFC 2332: NBMA Next Hop Resolution Protocol (NHRP) | RFC Editor

GETVPN

• One SA
• Any-to-Any
• No tunnels
  • Does not change the IPs
  • IPSEC Tunnel mode with Address Preservation
    • Works well with QoS or Traffic Engineering
• Replicates multicast well

HIPAA, GLBA, and PCI DSS all mandate encryption even over private IP networks.

DMVPN is OK, but requires an overlay, with additional complexity.

Point-to-point IPSec tunnels are poor at multicast replication, because the multicast must be replicated before it enters the tunnel.

Terms

GETVPN — Group Encrypted Transport VPN

GDOI — Group Domain of Interpretation

• Implements IKE

G-IKEv2

• Replaces GDOI

GM — Group Member

• All share the same crypto SA

KS — Key Server

TEK — Traffic Encryption Key

KEK — Key Encryption Key

• Control plane traffic

G-IKEv2

Message Exchange

sequenceDiagram
    participant GM as Group Member
    participant KS as Key Server

    GM ->> KS: HDR, initiator SA, initiator key exchange, initiator nonce
    KS -->> GM: HDR, responder SA, responder key exchange, responder nonce
    GM ->> KS: HDR, encryption and authentication, initiator ID, group identification, vendor ID
    KS -->> GM: HDR, encryption and authentication, responder ID, authentication data,<br/>sequence, group security association, key download

Migration Help

Walkthrough

References

Group Encrypted Transport VPN (GETVPN) Design and Implementation Guide

Security and VPN Configuration Guide - GETVPN G-IKEv2 Support - Cisco

Group Encrypted Transport VPN - Cisco

GETVPN Troubleshoot Guide - Cisco

RFC 9838: Group Key Management Using the Internet Key Exchange Protocol Version 2 (IKEv2) | RFC Editor

RFC 7296: Internet Key Exchange Protocol Version 2 (IKEv2) | RFC Editor

RFC 6407: The Group Domain of Interpretation | RFC Editor

uRPF

Loose mode Uses CEF to check for any entry on this device (not a default route).

Strict mode Uses CEF to verify the packet arrived on the interface the router would use to route back to that source.

References

Cisco - Unicast Reverse Path Forwarding Strict Mode - IOS-XE 17.x

Out of Band Management

Management Interfaces Should Never Be Accessible Via The Open Internet.

Terms

NMI — Network Management Interface

Dedicated Interface used exclusively for Administrative Access.

In-Band

The network for normal and routine traffic.

OOB — Out-Of-Band

A secondary network, not the routine path.

NMI Network

A OOB network that aggregates NMIs.

Jump Server

A special node approved to access in-band equipment.

Zero Trust

Assuming the network is already compromised.

Remote Hands

An on-site person, who can physically touch the equipment to plug and unplug cables.

VRF — Virtual Route Forwarding

A different routing table, not the global table.

Segmentation

Splitting the network into an in-band, and OOB network via VRFs.

In-Band Access Examples

• Internet
• Company Network

Out-of-Band Examples

• NMI network accessible via VPN only to Network Admins.
• NMI Accessible via Jump Server.
• NMI Accessible via Remote Hands.

References

BOD 23-02: Mitigating the Risk from Internet-Exposed Management Interfaces | CISA

SC-37: Out-Of-Band Channels - CSF Tools

Commission Delegated Regulation (EU) 2024/1774 – Article 13 (Network Security Management)

Point to Point Links

Terms

P2P — Point-to-Point

A link where the only devices on the network are directly attached to each other.

Ping Pong Attack

A device off-link sends a ping to an address between two routers.

This happens with v4 and v6.

Neither router knows about or has the address, so they forward it back and forth.

Point-to-Point v4

Number with ‘IP Unnumbered’ or use a /31.

Point-to-Point v6

Use a /127.

Config

• Layer 3 structures should go on ports, not SVIs.
• Use ECMP, not link aggregation.

Layer 3

Config that lives on the port can sense and respond to a link failure in 8 msec.

For an SVI to go down, it must be the only port, and it must sense the interface down, this takes 150+ msec.

References

RFC 3021 - Using 31-Bit Prefixes on IPv4 Point-to-Point Links

RFC 6164 - Using 127-Bit IPv6 Prefixes on Inter-Router Links

IPv6 Security Myths and Legends

Enterprise Campus Design - Multilayer Architectures and Design Principles - Cisco Live 2023

SSO

The owner of the control plane is the RP, the Route Processor. The Active RP sends and receives the hello packets.

The physical router-to-router connections terminate on the linecard. The linecard needs a FIB to pass traffic.

With multiple RPs, if one RP has a catastrophic failure, the other RP can take over (SSO + NSR) without dropping traffic.

To get zero packet loss during a RP failure, without notifying the peer or dropping any packets, these three technologies are required (SSO + NSR + NSF).

Terms

RIB — Routing Information Base.

This is where the RP stores its routes.

FIB — Forwarding Information Base.

This is the information necessary to program the linecard to pass traffic.

SSO — Stateful Switchover.

The RPs sync with each other and share state, (hopefully) enough state to prevent traffic disruption.

Checkpointing

All necessary information to perform the task is already on the standby RP.

Non-Stop Routing

The Control plane relationships and RIB are both checkpointed.

• AKA NSR

Non-Stop Forwarding

The FIB is checkpointed.

• AKA NSF

Graceful Restart

SSO/NSF/NSR are all vendor features that do no share state with the neighbor. GR is an IETF capability both devices must have turned on.

EoR — End-of-RIB.

This means the neighbor has shared the its entire routing table.

Graceful Restart

Restart Timer

If I drop the BGP session, Please wait this long before you stop forwarding me my traffic. (Default is 2 minutes)

Stale Timer

Once I send an open message, that means I’m working, so please give me this long before flushing my routes. (Default is 6 minutes)

Graceful Restart Mechanics

This is a BGP Example.

References

Cisco - Introduction to HA Technologies: SSO/NSF with GR and/or NSR

Graceful Restart Mechanism for BGP

Cisco - BGP Graceful Restart Per Neighbor IOS-XE 17.x

StackWise Virtual

This is a MLAG technology.

Used to reduce complexity from FHRP and STP.

Images courtesy of Cisco.

Terms

SVL

Stackwise Virtual Link

SVH

StackWise Virtual Header

DAD

Duplicate Active Detection

SVL

Between 1 and 8 inter-chassis links, for control-plane and data-plane traffic.

DAD

One worst case scenario, is the SVL fails, and both switches go Active/Active.

There are lot of bad effects from this:

• SVI IP Duplication
• SSH PKI Duplication
• STP Bridge ID Duplication

To help prevent this, it’s recommended to configure DAD links.

References

Products - Cisco Catalyst 9000 Platform StackWise Virtual White Paper - Cisco

Enterprise Campus Architecture

The C9000-L series, does not support Catalyst Center, and has lower stackwise Speeds.

Two Tier Collapsed Core

• The core and distribution switches are the same
• The center is running StackWise Virtual

Three Tier

Layer 2 Access With Traditional Multilayer

• Layer 2 is a single wiring closest, or access uplink pair.
• FHRP is used, but limits bandwidth to one uplink, vs both.

The Campus Network

• Campus networks are always oversubscribed.
• Over-subscription rates between 4-20 are common.
• Networks with over-subscription that results in queuing should implement QoS for voice traffic.

Core Layer

Fast and expensive.

Gear

• 9500
• 9600 (modular chassis)

Features

• No services
• Layer 3 only
• Always on
• Ideally, a minimum of 100G to conserve ports.

Distribution Layer Considerations

Purpose

• Aggregates wiring closets.

• Protects the core from high-density peering, and access layer problems.

• Summarize routes towards core

• Set STP root to be the FHRP Primary

• Enable

  • RootGuard on Downlinks
  • Loopguard on Uplinks

• Disable

  • DTP