Problem deleting tc rules

[Szymon Stefanek]

Hi all! :)

I see that this is partially covered in the mailing list archive
but at the moment I can't find a straight & working answer.

I have an imq device with dynamically attacched classes/qdiscs/filters.
There is a hashing filter that maps the last octet of an user's IP address
to a class (and associated qdisc). The "empty" filter looks like this:

filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16

I'm adding the hash entries dynamically, as users get attacched to the system
(this is a dynamic PPPoE access concentrator).

tc filter add dev imq0 protocol ip parent 1:0 prio 5 u32 ht
2:<LASTIPOCTETHEX>: match ip dst <IPADDRESS> flowid 1:<CLASSID>

<IPADDRESS> is the address of the newly attacched user, <CLASSID> is
a dynamically assigned unique identifier and <LASTIPOCTETHEX> is the
last octet of the user's IP address.

This works and I get a tc tree like this:

filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16

Nice.

If another user gets added to the same hash bucket I get something like:

filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 2:1:801 order 2048 key ht 2 bkt 1
flowid 1:4099 (rule hit 0 success 0)
match 0a060001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16


Now when the users go away I want to _programmatically_ remove the associated
filter entries. This doesn't work unless I manually specify the filter handle
(handle 2:1:801). The problem is that I actually don't know the filter
handle since its last part is assigned automatically by tc. I would need
to relaunch tc with the show option, capture its output and parse it...
this is an overkill. If I try to delete handle 2:1: (just like in the add
command) then ALL the rules with handle 2:1:* get deleted (and this is
obviously not what I want).

Trying to add filters with completly specified handle (like fh 2:1:4098)
doesn't work unless the last part is EXACTLY what tc would set automatically
(this is 800, 801 etc...). But since I have multiple unsynchronized processes
that do the adding job then I don't know which handles have been already
used and thus can't guess what which handle to add next. Synchronizing
the processes and sharing a database would be again a very huge overkill.

Trying to assign an unique filter id by using pref creates a mess since it
adds three filters at once for every preference value. And removing doesn't
work either with tc complaining about "No such file or directory".

I have read somewhere that deleting a qdisc will also delete the filters
attacched to it but this doesn't seem to work: the qdisc is deleted
but the filter in the hash bucket is not.

So finally, can I programmatically remove a filter without knowing exactly its
handle ? How ? Is there another way to match filters ? Maybe on flowid... ?
Add/remove by using direct syscalls ?

[Emmett Culley]

Post by Szymon Stefanek
Hi all! :)
I see that this is partially covered in the mailing list archive
but at the moment I can't find a straight & working answer.
I have an imq device with dynamically attacched classes/qdiscs/filters.
There is a hashing filter that maps the last octet of an user's IP address
filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
I'm adding the hash entries dynamically, as users get attacched to the system
(this is a dynamic PPPoE access concentrator).
tc filter add dev imq0 protocol ip parent 1:0 prio 5 u32 ht
2:<LASTIPOCTETHEX>: match ip dst <IPADDRESS> flowid 1:<CLASSID>
<IPADDRESS> is the address of the newly attacched user, <CLASSID> is
a dynamically assigned unique identifier and <LASTIPOCTETHEX> is the
last octet of the user's IP address.
filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
Nice.
filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 2:1:801 order 2048 key ht 2 bkt 1
flowid 1:4099 (rule hit 0 success 0)
match 0a060001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
Now when the users go away I want to _programmatically_ remove the associated
filter entries. This doesn't work unless I manually specify the filter handle
(handle 2:1:801). The problem is that I actually don't know the filter
handle since its last part is assigned automatically by tc. I would need
to relaunch tc with the show option, capture its output and parse it...
this is an overkill. If I try to delete handle 2:1: (just like in the add
command) then ALL the rules with handle 2:1:* get deleted (and this is
obviously not what I want).
Trying to add filters with completly specified handle (like fh 2:1:4098)
doesn't work unless the last part is EXACTLY what tc would set automatically
(this is 800, 801 etc...). But since I have multiple unsynchronized processes
that do the adding job then I don't know which handles have been already
used and thus can't guess what which handle to add next. Synchronizing
the processes and sharing a database would be again a very huge overkill.
Trying to assign an unique filter id by using pref creates a mess since it
adds three filters at once for every preference value. And removing doesn't
work either with tc complaining about "No such file or directory".
I have read somewhere that deleting a qdisc will also delete the filters
attacched to it but this doesn't seem to work: the qdisc is deleted
but the filter in the hash bucket is not.
So finally, can I programmatically remove a filter without knowing exactly its
handle ? How ? Is there another way to match filters ? Maybe on flowid... ?
Add/remove by using direct syscalls ?

I resolved this by adding "pref <LASTIPOCTETHEX>" to the filter rule:

tc filter add dev <iface> parent 1:0 protocol ip pref <user_id> u32 match ip dst <user_ip> flowid 1:<user_id (HEX)>

replacing "add" with "del" to remove filter.

In my case I used the last two octets to create a user_id value as I am serving DHCP to subnet 172.16.128.0/17

Note that the pref value has to be in base 10.

Regards,
Emmett

[Szymon Stefanek]

Post by Emmett Culley

Post by Szymon Stefanek
I have an imq device with dynamically attacched classes/qdiscs/filters.
There is a hashing filter that
[...]
So finally, can I programmatically remove a filter without knowing
exactly its handle ? How ? Is there another way to match filters ? Maybe
on flowid... ? Add/remove by using direct syscalls ?

tc filter add dev <iface> parent 1:0 protocol ip pref <user_id> u32 match
ip dst <user_ip> flowid 1:<user_id (HEX)>
replacing "add" with "del" to remove filter.
In my case I used the last two octets to create a user_id value as I am
serving DHCP to subnet 172.16.128.0/17
Note that the pref value has to be in base 10.

Hum. I have tried this.

Or better, my problem manifests when there are collisions of filters
inside a single hashtable bucket. Since the ht is hashing by last octet
then a single bucket can contain 2^24 ip addresses (the remaining octets).
I have then tried using (ipaddress >> 8) as preference value.

Here comes the first problem: priority values seem to be limited to 16 bits.
That is, if you add something with priority 0xaffff you'll end up with
real priority 0xffff which will collide with 0xbffff, for example.

The second problem is that if I use priority then I get a very different
filter layout. For each different priority used two additional filter
lines are printed by "tc filter show"...
The difference is between:

filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16

where priority wasn't used (and it's working) and

filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
filter parent 1: protocol ip pref 7 u32
filter parent 1: protocol ip pref 7 u32 fh 801: ht divisor 1

where a different priority was used.

Hm? Looks ugly.

Now, I'm not a tc expert but the output suggests that a complexier filter
hierarchy is created in this case and an additional "fh 801:" jumps
out from nowhere. In both cases the filter I've just added is the third line
of the listing: in the second listing it STILL has pref of 5! (????)

Since I tend to not trust stuff that I don't understand
at the moment I've choosen the very-dirty-but-at-least-undestandable solution
of using some grep & sed to get back the filter handle.

To add:

tc filter add dev @IMQDEV@ protocol ip parent 1:0 prio 5 u32 ht
2:@LASTIPOCTETHEX@: match ip src @IPADDRESS@ flowid 1:@TCCLASSID@

To remove:

tc filter del dev @IMQDEV@ parent 1:0 handle
$(
tc -s filter show dev @IMQDEV@ | grep 'flowid 1:@TCCLASSID@' |
sed -e 's/filter[A-Za-z0-9: ]*fh//' | sed -e 's/order.*//'
)
prio 5 u32

This forces me to spawn several children through a shell, is strongly
dependant on tc output (that might change in a future version) and makes batch
processing impossible...but at least it works and *maybe* I'll undestand it
in one year from now :D

...but if somebody comed out with a nicer solution I'd happily use it...

[Emmett Culley]

Post by Szymon Stefanek

Post by Emmett Culley

Post by Szymon Stefanek
I have an imq device with dynamically attacched classes/qdiscs/filters.
There is a hashing filter that
[...]
So finally, can I programmatically remove a filter without knowing
exactly its handle ? How ? Is there another way to match filters ? Maybe
on flowid... ? Add/remove by using direct syscalls ?

tc filter add dev <iface> parent 1:0 protocol ip pref <user_id> u32 match
ip dst <user_ip> flowid 1:<user_id (HEX)>
replacing "add" with "del" to remove filter.
In my case I used the last two octets to create a user_id value as I am
serving DHCP to subnet 172.16.128.0/17
Note that the pref value has to be in base 10.

Hum. I have tried this.
Or better, my problem manifests when there are collisions of filters
inside a single hashtable bucket. Since the ht is hashing by last octet
then a single bucket can contain 2^24 ip addresses (the remaining octets).
I have then tried using (ipaddress >> 8) as preference value.
Here comes the first problem: priority values seem to be limited to 16 bits.
That is, if you add something with priority 0xaffff you'll end up with
real priority 0xffff which will collide with 0xbffff, for example.
The second problem is that if I use priority then I get a very different
filter layout. For each different priority used two additional filter
lines are printed by "tc filter show"...
filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
where priority wasn't used (and it's working) and
filter parent 1: protocol ip pref 5 u32
filter parent 1: protocol ip pref 5 u32 fh 2: ht divisor 256
filter parent 1: protocol ip pref 5 u32 fh 2:1:800 order 2048 key ht 2 bkt 1
flowid 1:4098 (rule hit 0 success 0)
match 0a050001/ffffffff at 16 (success 0 )
filter parent 1: protocol ip pref 5 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 5 u32 fh 800::800 order 2048 key ht 800 bkt
0 link 2: (rule hit 0 success 0)
match 00000000/00000000 at 12 (success 0 )
hash mask 000000ff at 16
filter parent 1: protocol ip pref 7 u32
filter parent 1: protocol ip pref 7 u32 fh 801: ht divisor 1
where a different priority was used.
Hm? Looks ugly.
Now, I'm not a tc expert but the output suggests that a complexier filter
hierarchy is created in this case and an additional "fh 801:" jumps
out from nowhere. In both cases the filter I've just added is the third line
of the listing: in the second listing it STILL has pref of 5! (????)
Since I tend to not trust stuff that I don't understand
at the moment I've choosen the very-dirty-but-at-least-undestandable solution
of using some grep & sed to get back the filter handle.
$(
sed -e 's/filter[A-Za-z0-9: ]*fh//' | sed -e 's/order.*//'
)
prio 5 u32
This forces me to spawn several children through a shell, is strongly
dependant on tc output (that might change in a future version) and makes batch
processing impossible...but at least it works and *maybe* I'll undestand it
in one year from now :D
...but if somebody comed out with a nicer solution I'd happily use it...

Here's what I show (for one connection):

[***@lab1 ~]# tc filter show dev eth0
filter parent 1: protocol ip pref 65004 u32
filter parent 1: protocol ip pref 65004 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 65004 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:fdec
match ac13fef6/ffffffff at 16
[

This is with the last two octets (254.246 in this case).

I understand from the docs and much googling that the pref parameter is only to give priority within a class, but in this case each user has it's own qdisc and class rule. And it seems to be working.

I'd be happy to send you the entire configuration...

Regards,
Emmett