why shape incoming traffic

[Don Cohen]

qdisc (shaper) only for outgoing data. It is generaly
believed to be dumb to throttle data when they already
reached your computer or gateway.

The only reason I've ever been able to see for incoming data shaping is
to reorder packets so that, for example, FINs are sent to their
respective applications before SYNs (just an example) or ACKs before data.

It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

However, it does seem useful to be able to shape the incoming traffic
destined for the local machine.

For example, suppose this machine is running a server that you want
to limit to 10 connections/minute. It seems reasonable to do this
by limiting the rate at which syns are delivered to that server.
That might be a lot easier than trying to modify the server.

You might argue that doing it in the server would have the advantage
of being able to make more intelligent decisions about which ones to
accept and which to drop, but in fact the opposite could also be the
case. (I'm working on a project that provides an example.)

Similarly, as a way to limit resource usage, you might want to limit
the rate at which a server or client gets input (and sends output, but
you can already do that).

What I find frustrating is that, as a firewall, I can already do this
stuff for the servers (and clients) running on OTHER hosts, but I
can't do it for those running on the local machine!

[bert hubert]

Post by Don Cohen

qdisc (shaper) only for outgoing data. It is generaly
believed to be dumb to throttle data when they already
reached your computer or gateway.

The only reason I've ever been able to see for incoming data shaping is
to reorder packets so that, for example, FINs are sent to their
respective applications before SYNs (just an example) or ACKs before data.

It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

An important reason to be able to do incoming shaping is that you may lose
information while routing, which doesn't help you with shaping on egress.
For example, to shape traffic going via a proxy server, outgoing.

There is no way to recognize on ppp0 that traffic was generated by an
internal host who sent it to eth0, where it was packaged by a proxy, and
sent out over ppp0.

With ingress shaping I would be able to do that.

So in short - if you like CBQ, HTB etc on egress, and you accept that
informationloss occurs between ingress and egress, you want to be able to
shape ingress.

Regards,

bert

[Don Cohen]

Post by bert hubert

Post by Don Cohen
It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

An important reason to be able to do incoming shaping is that you may lose
information while routing, which doesn't help you with shaping on egress.

Isn't the usual solution to mark the packets at input and use the
marks for shaping? When is that not adequate?

[bert hubert]

Post by Don Cohen

Post by bert hubert

Post by Don Cohen
It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

An important reason to be able to do incoming shaping is that you may lose
information while routing, which doesn't help you with shaping on egress.

Isn't the usual solution to mark the packets at input and use the
marks for shaping? When is that not adequate?

Try getting your mark to survive squid.

[Michael T. Babcock]

Post by bert hubert

Post by Don Cohen
Isn't the usual solution to mark the packets at input and use the
marks for shaping? When is that not adequate?

Try getting your mark to survive squid.

Of course, one could spend a few months figuring out how to track marks
within squid so that outgoing requests have the same TCP options set as
the incoming ones ...

PS, 'months' being estimate based on prior readings of source ...

[Michael T. Babcock]

Post by Don Cohen
It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

Obviously ...

Post by Don Cohen
However, it does seem useful to be able to shape the incoming traffic
destined for the local machine.

... the only other option, besides rejection of course ;-)

Post by Don Cohen
For example, suppose this machine is running a server that you want
to limit to 10 connections/minute. It seems reasonable to do this
by limiting the rate at which syns are delivered to that server.
That might be a lot easier than trying to modify the server.

I use application-level software for that (iplimit in conjunction with
tcpserver allows me to limit per-service connection rates on a per-source
basis).

Post by Don Cohen
You might argue that doing it in the server would have the advantage
of being able to make more intelligent decisions about which ones to
accept and which to drop, but in fact the opposite could also be the
case. (I'm working on a project that provides an example.)

I agree with the Unix-way of doing things usually, not the emacs way --
don't build it into the program if it works just as well outside the
program; thus iplimit. Programs that accept their own connections,
like Apache, can't use an external program of course (yet), so it would
make sense to build this in although I proxy my incoming connections
through a Squid service set up as an accelerator with bandwidth pools
turned on.

Post by Don Cohen
What I find frustrating is that, as a firewall, I can already do this
stuff for the servers (and clients) running on OTHER hosts, but I
can't do it for those running on the local machine!

I've got around that in some situations by teaching myself to treat the
gateway/firewall box as a non-service box -- it runs nothing but the
firewalls, tunneling, forwarding and shaping. This allows it to work as
desired (all traffic goes out one of the two interfaces; none goes
to the local host).

[Don Cohen]

Post by Michael T. Babcock

Post by Don Cohen
It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

Obviously ...

Bert didn't think it was so obvious.
(I don't understand yet what squid is doing.)

Post by Michael T. Babcock

Post by Don Cohen
For example, suppose this machine is running a server that you want
to limit to 10 connections/minute. It seems reasonable to do this
by limiting the rate at which syns are delivered to that server.
That might be a lot easier than trying to modify the server.

I use application-level software for that (iplimit in conjunction with
tcpserver allows me to limit per-service connection rates on a per-source
basis).

There are benefits to doing this sort of thing at the firewall.
For instance, it works for servers running different OS's.
And once you have the control working for your inside network it
seems unreasonable to have to change the implementation in order
to move a server to the firewall machine.

(Another advantage of doing it at the firewall is that you can then
control the aggregate of traffic to different hosts. But the ability
to shape traffic to the local host would not be enough to retain that
advantage when moving a server to the firewall.)

Post by Michael T. Babcock

Post by Don Cohen
You might argue that doing it in the server would have the advantage
of being able to make more intelligent decisions about which ones to
accept and which to drop, but in fact the opposite could also be the
case. (I'm working on a project that provides an example.)

I agree with the Unix-way of doing things usually, not the emacs way --
don't build it into the program if it works just as well outside the
program; thus iplimit. Programs that accept their own connections,
like Apache, can't use an external program of course (yet), so it would
make sense to build this in although I proxy my incoming connections
through a Squid service set up as an accelerator with bandwidth pools
turned on.

I'm not sure what the emacs reference was all about. I'm not
suggesting building the limit into the server, just the opposite.
(My solution also works with apache.)

Post by Michael T. Babcock

Post by Don Cohen
What I find frustrating is that, as a firewall, I can already do this
stuff for the servers (and clients) running on OTHER hosts, but I
can't do it for those running on the local machine!

I've got around that in some situations by teaching myself to treat the
gateway/firewall box as a non-service box -- it runs nothing but the
firewalls, tunneling, forwarding and shaping. This allows it to work as
desired (all traffic goes out one of the two interfaces; none goes
to the local host).

That's a solution dictated by the necessity imposed by our inability
to shape traffic to the local host. Clearly there are times when
you'd rather run something on the firewall. There are also times
when you have to - programs that are supposed to communicate with the
outside world to control the firewall. It's just as important (maybe
more) to shape the traffic to those programs as to others.

[Michael T. Babcock]

I'd argue that this request is coming from the local machine and
should be classified as such. I gather it's not forwarding the
original packets.

You're correct that it is not sending the original packets, but if 300
users from 5 departments with bandwidth sharing limits are browsing the
web with one copy of Squid between them and the 'net, its very nice if
Squid's web usage gets taken into account when it has to fetch sites.

What I don't understand: is the user sending packets to the site with
the original web page or to the squib server? If the original site,
then how is the squib server getting them?

That depends on your configuration; Squid can be set up as a transparent
proxy so that all requests made to given ports (80, 443, etc.) are forced
through Squid instead so that the user doesn't have the choice.

The point is that I want to maintain legitimate communication with
these servers when someone is trying to attack them.

If all you want is to have legitimate use, set up a class for interactive
traffic, reserve it some amount of bandwidth, cap it at another amount (to
keep a user from using SSH to eat all your bandwidth) and add SFQ to it to
make sure no one connection gets all the bits.

My new Internet gateway box is actually set up without any login or port
services except forwarding and firewalls -- there are no listening ports at
all and no way for me to log into it except over serial port or at the
terminal. The serial port is connected to an external modem; logging in
is therefore "out of band". Any form of non serial login is assumed to be
an intrusion, but that's off-topic ;-).

[Michael T. Babcock]

Post by Michael T. Babcock
That depends on your configuration; Squid can be set up as a transparent
proxy so that all requests made to given ports (80, 443, etc.) are forced
through Squid instead so that the user doesn't have the choice.

So squid is intercepting packets addressed to somewhere else?
How is it doing that?

Usually through port redirection using your firewall (or ipchains ;-).

SFQ is not a good defense - the attacker just sends you random source
addresses and ports and now his packets have priority over yours
(which all come from the same address/port). But you're close.

That only works if traffic is generated on all of those hashed address/port
pairs in which case the attacker's data flow is just as stymied as mine.

[Don Cohen]

Post by Michael T. Babcock

Post by Michael T. Babcock
That depends on your configuration; Squid can be set up as a transparent
proxy so that all requests made to given ports (80, 443, etc.) are forced
through Squid instead so that the user doesn't have the choice.

So squid is intercepting packets addressed to somewhere else?
How is it doing that?

Usually through port redirection using your firewall (or ipchains ;-).

Ok, so I think it's reasonable to view the outgoing http client
packets as coming from the squid machine, and they can/should be
shaped/accounted as such.

That brings us back to my original argument that it's not necessary
to shape incoming traffic that is to be forwarded, since that traffic
can be shaped on the way out. This is not strictly true, since the
work at input could save some work between the input and output, but
it's a pretty good approximation. On the other hand, if you accept my
argument that it's useful to shape traffic to the local machine, then
you agree with me that it would be worth while to add shaping at
input, in which case the case above that I suggest is unnecessary
comes for free.

Post by Michael T. Babcock

SFQ is not a good defense - the attacker just sends you random source
addresses and ports and now his packets have priority over yours
(which all come from the same address/port). But you're close.

That only works if traffic is generated on all of those hashed address/port
pairs in which case the attacker's data flow is just as stymied as mine.

Attackers generally don't really want to communicate with you. They
just want to prevent others from doing so. In any case, I'm trying to
protect legitimate communication even when under an attack by that
sort of attacker.