Re: [e2e] opening multiple TCP connections getting popular

From: Bob Briscoe <rbriscoe(at)jungle.bt.co.uk>
Date: Thu Aug 30 2007 - 13:59:17 EDT

Joe,

At 16:57 30/08/2007, Joe Touch wrote:

>Lachlan Andrew wrote:
> > Greetings Joe,
> >
> > On 30/08/2007, Joe Touch <touch@isi.edu> wrote:
> >> Lachlan Andrew wrote:
> >>> I thought that Bob's suggestion was simply that the transport/network
> >>> layer provide a mechanism for charging based on the congestion caused.
> >>> That sort of mechanism *is* "solvable at the transport layer".
> >> If we charge based on congestion caused, that's a network layer issue,
> >> not transport.
> >
> > True, the charging is network layer, but letting the application know
> > how much it is being charged so that it can change its rate is
> > transport layer, isn't it?
>
>It's transport layer if you're charging the app based on what the app
>sends to the transport. Unfortunately, the network can be just as
>congested by a retransmission as a first try, so this is network
>pricing. Although the transport and application may _react_ to it, it is
>generated (and thus the problem exists) at the network layer.

Here's where the transport layer comes in: Policing. Even tho it's "in the network", policing is a transport layer function.

We have proposed a bulk per-user policer based on measuring downstream congestion volume (basically a big token bucket of downstream congestion tokens for all flows at once). Despite being bulk across flows, this policer incentivises each flow's end-point transport to take care to manage congestion between themselves within the envelope of the bulk policer. But just because the policer is bulk, doesn't mean it's not transport.

The subtle point here is that I haven't actually proposed charging for congestion. Strictly I've proposed how to reveal a generally useful metric at the network layer - downstream congestion volume. Certainly that /could/ be charged for, but that wasn't my motivation... at least not for end-users (because there's loads of evidence users don't like unpredictable pricing etc etc).

Instead, we wanted to reveal a metric in network layer headers that would allow inline transport mechanism (policing) to be applied to it, not just out of band (charging, penalties etc).

(Hence the need for a metric about the expectation of downstream congestion so it could be policed at the ingress before the packet entered the internetwork.)

However, between networks I was motivated by congestion charging or any similar use of the metric (e.g. incorporation in inter-domain SLAs with penalties for exceeding thresholds etc). Strictly, that's application-layer (in the management-plane if that's how you see the world) on top of a network-layer metric.

> >> First, however, what does that mean?
> >>
> >> There are many issues there; it's not cut-and-dry.
> >
> > Agreed. The question is whether the IETF should be working to address
> > those issues, or ignoring them.
>
>I've stated before - the IETF isn't the IRTF; some of this is research
>(i.e., what are the dimensions of the solution). Some of it is also a
>combination of policy and pricing - neither of which are IETF purvue either.

Of course I'm not expecting the IETF to do the policy, but I am expecting it to provide the protocol that can have policy applied to it. That's very much IETF land.

>The IETF is a good place to standardize a technical solution once we
>know what we're solving. At this point, "we need to do something" is
>insufficient rationale to proceed

"We need to do something" is a rather dismissive characterisation of a fully spec'd protocol proposal:
<http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-re-ecn-tcp-04.txt>

I can't win either way. When I put re-ECN to the IETF, everyone didn't understand why it was needed. Now I've explained why it's needed, people complain the IETF doesn't get involved in economics and policy. I'm not asking the IETF to get involved in economics and policy, I'm asking for a bit in a header. The wider industry does the policy. But the IETF needs to state the requirements it understands it is trying to meet.

>(and, as others have noted, it's not
>clear we _need_ to do anything).

Tell that to the CEO of any network operator. I think you're saying there's not an engineering problem. But that's because resource allocation problems are economic problems not engineering problems... However, this one has an engineering solution. Of course, the network engineering continues to _work_ with any particular sharing of resources, but that doesn't mean it's 'working' in an economic sense.

Consider this: folks at the IETF can expend weeks on whether a certain sentence about the minutiae of a certain protocol field will be a MUST or a SHOULD. But apparently we don't need to spend any time at all on fixing the whole of resource allocation and accountability.

> >> A solution inside the transport layer solves only the most
> >> trivial variant, and leaves so many loose ends elsewhere it doesn't put
> >> a dent in the overall problem.
> >
> > Can't the same be said for enforcing flow-rate fairness?
>
>Excellent point. You'll note that we do nothing to enforce that. The
>goal is that the network congestion feedback is proportional the number
>of streams - there's no rule that prevents opening multiple streams (as
>was noted) or merging them either. The whole thing works mostly because
>the degenerate case - a stream per packet - is prohibitive.
>
> > If that does
> > more than put a dent in the overall problem, then congestion charging
> > can make a bigger dent. If flow-rate fairness *doesn't* put more than
> > a dent in the overall problem, should we forget about "TCP
> > friendliness"?
>
>We don't enforce that either. We encourage that as a general principle,
>and assume (as above) that so long as the number of TCP streams is
>roughly proportional to the number of users, things will work fine.

Server farms would get a couple of flows? I think you might mean "roughly proportional to the amount different users are paying for their access". But then, why constrain ourselves so that usage has to be approx proportional to access link capacity? It's much more economically efficient to separate the two:
- access capacity is only about peak bit rate - different users have extremely different demands for how much they use their capacity

>They
>don't work if the number of flows grows with the load; if we get to a
>point where that actually happens, then maybe we'll need to do somthing.

Indeed, we have got to that point.

And what about if the duration of the numerous flows from single users are also hundreds of times longer than those from others (who are paying the same)?

Indeed, we have got to that point too.

See for instance any graph on distribution of usage between different residential users. E.g. Figs 12 & 13 in Kenjiro Cho's paper on p2p usage in Japan in SGICOMM'06.
<http://www.sigcomm.org/sigcomm2006/discussion/showpaper.php?paper_id=21>

But anyway, why is it that the IETF can happily expend time on volumes of engineering trivia, but an economic efficiency problem is given a bar hundreds of times higher before the IETF should stir itself into action? If there was a 54% inefficiency in a protocol, the IETF would be in there trying to fix it. But if it's 54% economic inefficiency it's not important?

Bob

>Joe
>
>--
>----------------------------------------------------------------------
>Joe Touch Sr. Network Engineer, USAF TSAT Space Segment
> Postel Center Director & Research Assoc. Prof., USC/ISI
>

---

Bob Briscoe, <bob.briscoe@bt.com> Networks Research Centre, BT Research B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK. +44 1473 645196 Received on Thu Aug 30 14:49:09 2007