MAT Working Group session
RIPE 87
28 November 2023
4 p.m.
MASSIMO CANDELA: Hello. Here we go. So, good afternoon everybody. My name is Massimo Candela and welcome to the measurement analysis and tools. This time in Rome. So, I'm really happy about this. I was born and raised in Rome, I moved to the nerds at 26, so Rome is my home town, and after many years of participating to the RIPE meetings, I am really happy to see so many familiar faces in my home town. So, welcome to Rome for a definition of it but still welcome to Rome.
And as I said, this is the measurement analysis tool Working Group. We have a really nice and packed agenda so we will proceed. We are also a few minutes already late but we should have have plenty of time.
So these are the MAT Chair Working Group and there is Nina and there is Stephen. We will alternate to Chair this session. And this is a view of the agenda. So, we have some interesting talks about censorship, about traceroute, plenty of traceroutes, and also we will see performance of QUIC and we will finish with the RIPE NCC tools update.
And I think we can immediately start with the ‑‑ closing agenda.
STENOGRAPHER: I like it. Let's go.
MASSIMO CANDELA: We can immediately go with the first presenter. The first presenter is. He founded the Open Observatory of Internet only in 2011. He previously worked on the Tor project and created several free software projects that promote human rights such as global leaks. He co‑founded and serves as the vice‑president of centre of human rights. He studied mathematics and computer science at the University of Rome. So, another from Rome.
The stage is yours.
ARTURO FILASTO: Thanks very much. It's great to be here in Rome with you all to tell you a bit about the work that we're doing at OONI. I had lender about the MAT, you know, measurement analysis and tools Working Group of RIPE when I met in Montreal, Stephen, and it seemed like an excellent opportunity to come here and tell you a bit about our own MAT, the measurement aggregation tool kit. So, you know ‑‑ maybe I only find this funny, but I thought it would have been a nice joke.
So, any ways, I will tell you what we are going to talk about today, and the main topic is the area of work that we focus on, as OONI, the open observer tree of network interference, and what we do is we measure Internet censorship. So, before going deeper into that, it's maybe useful to clarify what we mean by Internet censorship. So, when we talk about Internet censorship, what we're talking about is the intentional control or suppression of what can be accessed, published or viewed on the Internet.
So, a key part of this is intentional, right, and why this is challenging is that often times I guess many people even in the MAT Working Group face it when doing network measurement, you deal with certain things that you observe as somebody carrying out Internet measurements, and you don't necessarily know what is causing that behaviour. So, for example, we deal a lot with, you know, this level of uncertainty where you see some signal that could be attributable to Internet censorship, but it might also be attributable to some sort of transient network failure, some sort of error condition that's happening on the network and is not due to an intentional restriction.
Before we go deeper in that, I think it's important to also say why we measure Internet censorship, why is this important. And I think it's important for a variety of different reasons. For starters, it's really important because, you know, the work that we do at OONI, it's in support of a lot of advocacy efforts aimed at fighting Internet censorship. We see ourselves as a provider of data to support evidence for these types of fights. (Add case) we feel it's important to offer an important archive on Internet data worldwide so that we can study this over a long period of time and understand how it has evolved. It's also really important to have oversight and transparent so that those implementing Internet censorship are accountable and that, you know, we know what is blocked and how it is being blocked.
We also support a lot of research interventions that study Internet censorship from a technical perspective. But also efforts aimed at making the Internet more resilient to forms of censorship, so we support a lot of Internet censorship circumstance couple convention tool developers in making their tools more robust against the evolving forms of Internet censorship.
This is why it's important. But, you know, what is the real impact of censorship? When does it happen?
.
I won't go too much into all of these, but, you know, just to give you an idea of when censorship happens, like what are the triggers for it, and so I think broadly speaking, censorship tends to happen when there are, you know, there is a baseline of censorship that is implemented basically everywhere around the world, but then we also see an increase in censorship around certain political and social events, whether that is elections, so, we have seen, you know, over the years social media getting blocks during elections in many countries around the world. We see it during political transitions, when there was the coup in Myanmar, they started blocking WhatsApp, there were Internet blackouts affecting the country. We see it during conflicts. So, you know, when there was an increase in tension between Azerbaijan and Armenia, they started to block TikTok. We saw is obviously in the escalation of the conflict between Russia and Ukraine. But it also very importantly targeting marginalised communities. So we see a lot of groups, whether for certain idealogies, beliefs or religions, or just attributes of their being, getting impacted by the restriction in accessing certain types of content on the Internet.
And lastly, we see it around protests. So like, you know, one case that I will now dive a little bit deeper into is, you know, in Iran last year, there were, there was a wave of protest following the mass Armenian protest. We saw a significant intensification of the forms of censorship being deployed inside of the country.
And so, around that time we worked on a multistakeholder report investigating the various forms of Internet censorship that followed the mass or mean a protest in Iran. This was a joint project between OONI, measurement lab, CloudFlare, calm tech, ISOC and article 19. It was facilitated by the European Commission and the United States government, and basically what we did is we took all of these different datasets and we used them to be able to study how Internet censorship had been, had escalated basically in the country following the protest. And I just wanted to pick out a few bits of information that I think are particularly interesting and relevant to this type of audience, but you can obviously find the full report on our site.
So, in terms of the timeline of events. As, you know, probably, you know, Mahsa Amini was murdered on the 16th September 2022, and this triggered, you know, sort of not unprecedented but a significant wave of protests in the country, right. Protests emerged in many different regions, and following this enormous wave of social uprising and protests, the government started cracking down on the protesters, and obviously in the physical space but also on the Internet. And so we saw a significant increase in the forms of restriction to access to the Internet.
Specifically, one thing that we saw that had not ‑‑ that we had not previously seen was the use of digital curfews is how we were calling them, which is something that we had seen before in Myanmar, in Iran that was the first time that saw this type of technique being used which was basically for specific hours during the day the Internet was completely disconnected. So there was no access. And this was limited either to specific regions or in some cases it was nationwide, and access to the global Internet was restricted for several hours. And this was done over the course of 13 consecutive days, and here we can see, you know, the presence of this curfew in the iota data for example which monitors Internet blackouts.
Another thing we started to see was that several social media platforms that previously were accessible started to be blocked.
Similarly, app stores started to get blocked as well. Probably as a result of people trying to use circumstance convention tools to get around the restrictions and so they were basically cracking down on different ways that people would use to gain unrestricted access. Similarly browser extension repositories started to be blocked. And then a few things interesting to you is that encrypted DNS, which previously was mostly working in Iran, started to be restricted much more heavily.
So, we saw all of the main encrypted DNS providers being blocked. And this was, you know, as you can see from these charts, previously they were somewhat working, some of them. And then, you know, from around ‑‑ this is October ‑‑ end of September, October, they started to get entirely blocked.
Other interesting things that we saw was that HTTP 3 and QUIC blocked only on certain networks but it was like the whole protocol was entirely restricted. We do not know why QUIC and HTTP 3 was specifically targeted. We speculated that it might be as a result of their censorship capabilities not being sophisticated enough to implement restrictions on these newer protocols.
On some networks, we also saw complete block of IPv6. So, you know, there is, there was a presentation before like talking about adoption of IPv6. In some places it is blocked, like they decided to entirely block it. And so, you know, this is stuff that I guess has an impact on the rollout of new technologies that are more secure and in some cases an improvement, but if they do not work, then that is quite unfortunate.
So, this is an example of the research that we do. But we really see ourselves as an enabler to conducting this type of research independently. So OONI was a project started back in 2011, 2012, and our goal is basically to empower decentralised efforts at understanding network interference worldwide. We really would like to enable people to use our data, use our tools to do this type of research on their own. And so as a result of this, we have this open dataset that counts billions of network measurements from every country in the world and it's just out there for anybody to use
.
And the dataset is collected through our app, collected through OONI app, which you can install on basically any device.
But we also have a MAT. So, the MAT is the measurement aggregation tool kit, which is this open data platform that you can use to explore this giant data set, which is OONI, so that you can come up with your own queries of what kinds of network interference you might be interested in exploring.
And one of the things that I thought is like a reflection to make for this audience, is that, you know, there's kind of this challenge in presenting rich, technical information to non‑technical audiences, which is the audiences that we are mostly dealing with, in that, you know, as I was saying before, there is this margin of uncertainty where you have some number of data points that contribute towards confirming or refuting a certain hypothesis, but ultimately there is always some sort of margin of uncertainty. And sometimes the margin is not ‑‑ it's pretty clear. Sometimes the signal you see in the data is very obvious. Like, in this case when they started blocking WhatsApp in Senegal, you can clearly see that, before it's all green and then at some point for a brief period of time it's all yellow and then you know it starts being green again.
But other times it's not so clear. So this is like from today in Italy. RT.com in Italy. So as, you know, Russian news media is blocked in all of Europe, but not entirely blocked. Some networks do not block it; others do. And so, you know, all you can do is you can maybe break it down by network and you can drill down and see okay, can we remove the noise and try and look deeper.
And sometimes that is possible. Other times it is not. And so, this is an example where we were looking at the blocking of these RFERL sites during the Kasak elections and the signal was not really clear so we had to take steps in order to analyse it further. And basically, as part of this, we were able to confirm basically that in this case it was not a complete block but it was a form of throttling. So, what we did here was that we were looking at the timing information related to TLS hand shakes and what we noticed is that we used as a baseline a site that was hosted on the same provider as the sites that we suspected to be throttled. And then, you know, sort of drew that baseline and said okay, we expect the time to perform a TLS handshake to be this. But then, you know, we see in the data points for the specific news site to be significantly higher than what, you know, is our baseline.
And so through this we were able to effectively provide pretty strong evidence that it was intentionally being restricted because there would not be any other reason why you would see such high TLS handshake times for only the specific site and not another one that would have the same network path.
This is only the starting point. I invite you to check out all the resources we have on our website, and get involved in our community. We have a public SLAAC channel, and I'm happy to answer any questions you may have.
Thank you.
(Applause)
STEPHEN STROWES: We're running fairly close to time so if we can be fairly short on questions. But name and affiliation please.
SPEAKER: Max medium. We are Internet service provider trying to provide Internet access without any restrictions. Is there some solution for us that can be used to know about some of our assumption, just to change the route and use clean route that source is available? So what signals to us that something goes wrong for this place and we should change upstream to reach this.
ARTURO FILASTO: You are asking if there is something you can do that your upstream is implementing some sort of restrategic of the running OONI could help you collect this information. One thing to keep in mind is that, you know, we rely on testing lists. Like we cannot test all the sites and all the resources. So, we have sort of manually curated lists that we work with regional experts on, so that we have, you know, like a global one that we test everywhere and then every country has their own that you are kept up to date. But sometimes we might not be able to say ‑‑ like we can say if something is blocked and we see it that it is blocked. It is a bit harder for us to say nothing is blocked, you know. Because we do not measure everything. But, you know, to some extent, you can definitely run a tool like OONI and it might provide you some useful insight. But also do each out to me and we can discuss this further.
AUDIENCE SPEAKER: I have a question. What do you get information about what to check the list? Do you track any countries or some countries who have own block list? You may get it maybe from some open...
ARTURO FILASTO: Yeah. So we also do monitor block lists. So like you know, in Russia there is Roscom NATs or that has an official list and you can sort of download it. But as I was saying before, you know, the Roscom NATs err list is I think tense or hundreds of millions of sites, so we cannot realistically scan all of those from mobile phones of our users. So we need to do some sort of selection of what is relevant to test and what is not. That's why we mostly rely ‑‑ like we say our testing lists are not block lists, they are sort of a hand picked selection, representative selection of content that, you know, might be blocked but also might not be blocked and that if it were to be blocked, it would have a significant impact on the country. And so, you know, we have a network of partners ‑‑ organisations that we collaborate with to update these. There is also a specific group called net a list Kay who contributes researchers in a specific region to update our lists, and so that's sort of the process around that.
STEPHEN STROWES: All right. Thank you all. Thank you again.
Okay, up next we have Luca Sani. He received his masters in computer engineering from the university of Pisa and his PhD from the school of advanced studies of Lucca, with a thesis focused on the analysis of the in ecosystem. From 2014 to 2019 he was a researcher at the Italian national research Council where he worked on the Isolario project, which I know many folks are familiar with. And he joined catch point in 2019.
LUCA SANI: Hello everyone. Thank you for your introduction. I think you said everything I just want to add fun fact about me that I'm Luca and I am from Luca. This usually fun for people living abroad, so I apologise for Italian people here.
So, that said, this talk is about traceroute, which is probably the oldest network diagnostic tool in this environment. It was firstly implemented by Jacobson in late 80s, to answer the frustrating question about where are the packets going. So during the years, many different kinds of traceroute implementations have been done to run special different operating systems. But also to address slightly different kind of problems. For example, there are traceroutes tailored to multipath tracing, solution, detection. I think everyone in this room knows for example the Paris, Dublin and Pamplona traceroute.
In our company, releverage the famous Dmitry Pietrosanta Linux traceroute, which is very fast because it can send multiple probes at once. But most importantly it is open source and it is easily extendible.
So, during the years we announced this tool to offer new capabilities and I'm going to present you a few of them hopefully you can find them useful.
The traceroute announced its open source, so it's available to everyone.
So, following the tradition to name traceroutes after where the main developers reside, we decided to name it this. Pet reSanta is on the sea side but also is the location of a catch point of the Italian office.
So, these are the announcements that I want to talk about today. Let's start from the support for QUIC.
In particular, the possibility to run a QUIC traceroute. QUIC can be considered by all means a transport layer protocol even if it runs over UDP, because over UDP it offers a mechanism like congest and control, session establishment, encryption. So, it may be legitimate to think that QUIC packets may experience different behaviour in the network in respect to meaningless UDP packets like the one sent by normal UDP traceroute.
So, in this mode, our traceroutes sends QUIC initial packets. Packets that are used in QUIC to establish a session. These packets, as in normal traceroute behaviour, have incremental TTL, so probes expiring on intermediate objects will not trigger an error, which allows us to track the pathrd to our destination. When an initial packet hits the destination, we may have different possibilities, depending whether the destination speaks QUIC or not.
If the destination speaks QUIC, it may reply with a QUIC initial packet and in this case we reply back with another initial packet saying that we won't continue the handshake. So we want to explicitly interrupt the session.
If the destination doesn't speak QUIC or maybe doesn't speak QUIC on the port we are targeting probably we could receive an isn't port unreachable. We still empty because we can't conclude the traceroute. In most case we won't receive any reply back. In this case the traceroute will conclue with a time‑out.
So the nice side effects of this traceroute is that, first of all, you can check if the path from the source of the destination is QUIC or if for any reason is filtered. Also, you can check if the destination speaks QUIC. And another thing that you can check is if the destination supports the explicit congestion authentication mechanism over QUIC. This is possible because with this traceroute you can set the ECN at the IP level inside the probes that we send and we can check the ECN counters in the QUIC reply.
I won't go into more dataset of that for the sake of time, but also I saw that there will be a talk later today about ECN over QUIC so I don't want to spoil that.
To summarise the algorithm that we use to perform a QUIC traceroute. They say requires a little TCP half open technique. In TCP you send a SYN. If it's in the SYN ACK you something the session with a reset.
Another announcement that I'd like to talk about is called TCP InSession. This is related to TCP traceroute. In classic TCP traceroute, each probe is a SYN packet. This means that when you do a TCP traceroute in a very short period of time, you send a lot of SYN packets to the destination, and these are at least you may different packets. First of all the difference in packets may legitimately follow different paths to what the destination. And these could be not a desirable behaviour if you want to achieve for example consistency within a single traceroute measurement.
Also, sending many SYN in a very short period of time may trigger firewall protection rules, because it may be seen for example as a SYN flow attack.
So we tried to overcome these problems by implementing TCP InSession. In this mode we first of all open our TCP session with a destination and then we send traceroute probes which are TCP data packets with one byte payload and incremental TTL. This is inspired from TCP side again, but differently from side car where they send duplicate data inside a previously opened TCP session. We explicitly open the session for the sake of doing the traceroute.
So, let's say that we have this set‑up. There is the source and the destination which is two hops away and we send to parallel probes per hop. So the first thing that we do is to open the TCP session in usual way with the three‑way handshake. Then we start sending the TCP data packets within the open session. The first thing that you may notice from this slide is that the sequence number that we use is not the expected one, but we explicitly insert a gap inside the sequence number stream because we don't want to bother the destination application. So, we are ‑‑ since we don't know in advance how far the destination will be, starting from the very first TCP data packet that we send, we insert a gap. So we are sure that when the destination receives the packet, the TCP stack in the destination will not deliver the data to the application because it is incomplete.
So, when the data received in the destination, the destination will reply with an ACK. Now, the problem with inserting a gap inside the sequence number stream is that the ACK generated by the destination will hack the same sequence number which is the very first sequence number that is missing in the sequence. So we are not able to match properly the ACKs with the probes that trigger the ACK. So we have a computer proper round‑trip time.
To overcome that in this mode we require the destination to support the selective acknowledgment mechanism. This is negotiated during the TCP handshake, and if the destination supports that, each ACK that is generated will include additional information about the probes that were received by the TCP stack destination when the ACK was generated. So in this way we can match the ACK with the sending probe.
The algorithm is a little bit more complicated than that, so I encourage you to look at the block that we wrote about that.
So, to summarise, the concept of TCP InSession mode it is that first of all it requires the destination to support the selective acknowledgment mechanism. This should not be an issue as of today but you don't know.
Also it works only if the destination is listening on the port that we are targeting with traceroute. Something that is not required by classic
.
Another thing that we actually open a session with the destination. For the time of the traceroute we assume some traceroutes on the destination.
The pros are we send just one SYN packet per traceroute. We avoid the problem to be seen as a SYN flow attack.
And also, traceroute probes are seen as part of a data flow. So, they are more likely to follow the same path from the source to the destination.
This is an example of the difference between a classic TCP traceroute and TCP InSession.
On the top left of the slide, you have a classic TCP traceroute. We are using the standard Linux traceroute configuration so we send three parallel probes her hop. You can see that for each hop you have multiple IP addresses replying means that the different SYN packets follow different paths. Also you may notice that the destination replied to only one out of three probes. So, it seems that we have some packet loss.
On the bottom right, you have the TCP InSession version. And you can notice that all hops on, most all hops have one hop, one IP address replying. And also the destination replied to all our probes. So, this is useful to achieve, let's say, more consistency with the measurement.
The last announcement that I'd like to talk about is the possibility for traceroute to work into the Azure environment. So, the context is the following:
.
Let's say that you have a virtual machine, a Linux virtual machine with a private IP address hosted by Azure Cloud. So, you make sure that inbound ICMP packets are allowed, but despite that if you try to run a traceroute, you get all asterisks for all the intermediate probes after the destination. So we investigated that and that happens because there is a mismatch at the IP layer, between the content of the layer at the probe that we sent in respect of the content that of the one that we receive.
In the slide on the right you can see the probe that is sent. In this case, this is IS MP request because we are talking about an ICMP traceroute but this affects all flavours of traceroute. In particular, you can see that the source IP address of the probe is the private one but if you look at the payload of the ICMP TTL exceeded, as you know you expect to see in the pay loud the original probe that goes in the expiration, you see that the source IP address of the probe is the public one. So, also, please notice that the destination address of the ICMP TTL exceededed is the product one. Somewhere there is a missing translation back between the public IP and the private IP in today encapsulated probe.
We asked Microsoft why this is happening and they simply said that if traceroute is not working, you should use ping. This was not like a good answer because we want really to use traceroute.
So we found a work around to that. We introduced the so‑called loose match mode inside traceroute. In this mode we open an additional ICMP socket and with this socket we are able to receive every ICMP message that is transiting on the network and we do the checks, the kind that we do to deliver the packets to the application layer directly inside traceroute. So we check everything except the source address of then encapsulated probe. Despite that we are pretty sure that we have enough information to be sure that the ICMP error is related to the probe that we sent.
For example in the ICMP case, we have the identification number, the sequence number, also the destination address of the probe.
So, there aren't many more announcements that we did. The important thing that this version of traceroute is completely open source, like the original one. Also I want to thank Dmitry for the work that he did in the past. So, feel free to contribute, to check the code, to open issues, to ask anything. And if you happen to be around, feel free also to come by and meet us in the office.
Thank you.
(Applause)
NINA BARGISEN: That was brilliant and interesting. Do we have questions?
AUDIENCE SPEAKER: Hi, Alexander Asimov. Thank you for your report. I really enjoyed the idea behind the TCP traceroute. I wonder if there is an opportunity to see multiple path by running it but not running it multiple times?
LUCA SANI: Do you mean if you run multiple times the TCP InSession?
AUDIENCE SPEAKER: So the advantage of the speed traceroute, that you can see all kinds of paths to the destination.
LUCA SANI: It depends on the use that you want to do with traceroute. If you want ‑‑ like the consistency along all the measurement maybe you are more interested in one thing, if you want to do multipath tracing, yeah you are absolutely right, classic TCP traceroute.
AUDIENCE SPEAKER: I am thinking that maybe this way of traceroute can be even improved if you run multiple TCP sessions simultaneous.
LUCA SANI: Sure.
AUDIENCE SPEAKER: And give an output showing different paths that were discovered with this session. For example, I have a flag that says please run not one TCP session but five, ten sessions and give me the output of all these sessions.
LUCA SANI: Yeah, that's a good suggestion. And okay. We can work on that. Yeah. Thank you.
NINA BARGISEN: Anybody else? Excellent. Thank you very much.
(Applause)
.
Our next speaker is Valentin behind Rick. And research assistant at the Technical University of applied sciences Augsberg around working mainly on network measurements and network protocol design. And take it away.
VALENTIN HEINRICH: Thanks. Hi all. Today I'm also talking about traceroute. But this time in the reverse direction. It's about reverse traceroute. And another fun fact to start. We are also following the naming tradition and we will call this Augsburg traceroute from the place of origin.
So then we already had a lot of stuff about traceroutes so I will keep this quick. Traceroute it's often referred toe as the number one go to too for troubleshooting problems. This is quote from Richard Steinberg err. He often holds these talks about correctly troubleshooting. "As it turns out troubleshooting with traceroute can be quite challenging even though at this office it appears simple. The output is just an alliance of hops and the measurement of times.
So, we will see soon enough an example about such a situation that appears simple at the sure FAS but is actually complex.
I will talk about reverse traceroute. I want to define ‑‑ we want to define a protocol extension by we, I mean myself, and haul infinite err who also works on the project, and it's the co‑author of the draft, we want to extend ICMP to allow our message exchange to carry first traceroute messages to others to implement reverse traceroute from end point to end point on the Internet. We have it running online in the repository that you can see here and very soon there will be Debian packages available. It's open source, it's licensed under the GPL, so feel free to play around with it.
Let's start. Suppose you assume that you have a problem, you fire a press code and you get this output. Okay, so it looks quite simple if we have five hops in total and we start the router in Augsberg, we go from Munich, then two routers in Frankfurt and then finally we reach the destination. And we can see that there is a spike in the reported so many times starting at hop 4 and this increase in the run time. It also holds for the last hop. What would be your conclusion? Basically we have three different options. Option A, I mean I don't see a problem here. This is how it should be.
Option B would be well there must be something wrong between router C and B because that's where we can first observe the spike. And the last option would be well, we cannot really tell given this output alone ex he'd
.
Now if you go with option C, then you are correct, because as we can see by this topology, you cannot really tell by the output alone because the trace that you saw before maps one to one to the topology you can see here. So if you were to run a traceroute on the client on the left‑hand side, to what's the server on the right‑hand side, that's the output that you would get, and judging as an outside observer from the topology, we can see that there is no problem between router C and D. But in fact there is a problem on the return path.
And that's a major limitation of traceroute. Because while traceroute is only able to illuminate the forward path, to measure the forward path, the measurements for the forward path are still influenced by the return path. So, in this example, when the clients traceroute probes reach the outer, and they generate a time exceeded message. This is message will take the return path of over router E and F and not the forward path any more. The congested link in this example between router E and F will introduce the previously observed spike in the router round trip measurements and this may lead us to false conclusions. So there is a simple solution to fix this. We need to get entire picture of the network, and for this we cannot rely only on forward traceroute or classic traceroute. We need reverse traceroute as well.
So, reverse traceroute, that's not a new idea. It dates back to 1993 where it was implemented as an IPv4 option.
So the idea was we define an IP option, ween code the originator source that was inside this IP option and then we sent a packet to what's a destination.
Now, each router on the path to what's the destination that encounters this option, this IP option sends back a special traceroute to the originator. Now the originator only has to collect these special traceroute messages and is able to discover the complete forward path. The same holds for the reverse path.
When the destination replies and keeps the same IP option intact, and following the same manner, the originator can simply collect the issued pacts and also illuminate the return path.
And this is a very good solution but it has a few shortcomings. And one is how does a probable involved? We all know that when we need routers abroad for an implementation that deployment will get much much more difficult.
Also, IP options were employed and IP in today's Internet are frequently stripped or filtered, and there was also a potential attack where the originator's IP address in the traceroute option could be spoofed and has thus an amplification attack could be launched.
We want to learn from this. And we don't want to require any routers supported. We do not use IP options. And we also want to keep the attacks office for application attacks as slow as possible. And this led us to the following message exchange.
Now I already mentioned that we are using ICMP for reverse traceroute and the way it works is that the client sends one reverse traceroute request over ICMP to a server. Now, inside this request, the client can supply a configuration and the configuration determines what attributes the probe that will be issued back by the server to what the client will have.
So, specifically the client can specify do we want to use UDP, TCP or ICMP probe? The client can specify of course the time to leave, to which the traceroute probe should exceed.
And of course the client should be multipath. This means the client should be able to control a field that is relevant for a load balancing on to this Internet so that when a load balancer is present on the return path, the client can actually perform a multipath tracing, it can control on which path the probe should travel.
So then in step 2, after passing this configuration in the reverse traceroute request, the server assembles the probe and it sends it back in step 2 towards the client. In this specific example the time level of 2 was given. So at router E the probe expires.
In step 3 we get back the at the moment exceeded message and we can calculate the round‑trip time now between receiving the probes response in step 3 and sending the probe in step 2. We augment this information with the source advice of the probe's response and we deliver this information back to the client in a new ICMP reverse traceroute. And in this way a client is able to request reverse traceroute measurements probe by probe from a remote destination.
Basically, that's the message exchange. Now, I want to take a bit more into the datasets, specifically the code points.
But first of all, reverse traceroute is defined for both ICMP and ICMPv6. For both, the message typically starts like this. I mean we have a type, if we have a code, we have a checksum and ten them message specific data. The question we have to ask ourselves is: Do we want to use new types and new codes for this reverse traceroute messages or do we want to reuse an existing type?
.
And really, the decision of which approach we choose, it was done to which one does actually work on today's Internet. And for this, we test add few middleboxes.
So, ossification in the Internet is unfortunately quite common and the biggest contributor are middleboxes. This happens when a middlebox assumes that has complete understanding of how the Internet works today, and when it sees announcing packets it just enforces a restrictive behaviour towards those packets which means that unseen packets, unknown packets, can be stopped by these middleboxes which makes the implementations of new protocols really, really hard.
So, we tested 12 different NAT implementations which were typical home gateways and we wanted to figure out which option works, option A or option B.
So, we first started by sending ICMP trust packets to these middleboxes. So, we start with option B first. And we did not send regular ICMP request packets but we used a new code, Codes 1 and 2 respectfully. If these requests, they made it not implementation, then we followed up with the response and checked if the response with the magic code was also able to traverse the NAT implementation.
And you can see the first two lines of this table that the results were actually quite promising. So, 11 out of 12 of these NAT implementations, they actually correctly forwarded both the requests and the responses with code 1 and 2 correctly.
Now, only a single implementation actually it worked a response.
Now, if we have a look at the results for the unassigned types which would be types 7 and 252, we get a different picture. Because in this case, only a single middlebox correctly forwarded the traffic.
Now, the majority of these NAT implementations, bypass is interesting because a bunch of these NAT implementations they actually forwarded the traffic, but they did not perform network at first translation, so they sent out a packet and a left a private source address in place. So this makes no sense at all, but yeah...
.
In the previous measurements, from the previous measurement, we can conclude the three using the ICMP types and extending those with new codes is good for deployability. On the other hand, using unassigned types will fail at the home gateway.
With regards to deployability we can forget this case. But just because these ICMP us types with the new codes to home gateways does not give us any guarantee that these packets will support an Applicant or not. So figure this one out, we performed another measurement.
And this time we picked 10 million IPv4 addresses at random, and we pinged each one of those. And the ones that pinged back were the ones we kept in the list. For each host that responded we then sent another ICMP echo packet with now the new codes, code 1. We collected the responses and we categorised the responses as shown by the table below. Now, filtered means that we did not get any response to this echo requests with code 1 back. This is the case in about 4% of the total collected responses.
So, reflective means we did get a response back and that the responses code matches the requests code. And this is really good news for us because what this means is that both the echo request and the echo response with code 1, they made it good public Internet and filtered and unaltered in the vast majority of cases, which means we can use these echo types with code 1 well to deploy it in the reverse traceroute messages. That's the case in over 92% of the cases.
Unreflective means that we got back a response but it was really just a typical ICMP echo response with code 1. And erroneous responses are response that is we cannot put in any of the previous bins.
So, overall, the conclusion that we can take away here is that we have to use the existing types, extend with a new code, and this is exactly what we are doing in our reverse traceroute draft, where we use ICMP echo and extend it with a new code.
So, as a conclusion. If you are interested, then you can read the draft. It's online. You can join the discussion at the Working Group mailing list. Any feedback will be welcome. If you want to get even more involved you can offer to host a reverse traceroute end point. As mentioned the code is online, and we will have packages for ininstillation. You can use our reverse traceroute client, which is all the multipath capable and it renders the multiple different paths to a graph. And you can send us the output. And what of course would be a huge boost, if we could talk about reverse traceroute as being part of Atlas in the future.
Thanks for your attention. That's all.
(Applause)
MASSIMO CANDELA: Thank you very much. We have a question.
AUDIENCE SPEAKER: Thank you for your talk. A couple of questions. First question is: Don't you feel that you are using RPC call on top of ICMP transport?
VALENTIN HEINRICH: Can you ask it again?
ALEXANDER AZIMOV: I have a feeling that you are reinventing RPC with ICMP transport, you are making requests to execute something on another device but using ICMP for this purpose.
VALENTIN HEINRICH: Actually, no, that's not what we're doing. We want to extend ICMP to function like ping but implement this reverse traceroute functionality. We don't want to this remote procedure as you call them.
ALEXANDER AZIMOV: Another question. What about spoofing and how the service discovery should be working?
VALENTIN HEINRICH: Service discovery, it's quite easily implemented. The client sends an invalid time to leave notice request and the server then responds with a packet that be be easily distinguished from a typical ICMP echo response. And so in this way the client can figure out that this end point is actually running a reverse traceroute instance.
ALEXANDER AZIMOV: Okay, and what about spoofing?
VALENTIN HEINRICH: We don't have any counter measures for spoofing as such. Well, we rely on source validation but the next release for reverse traceroute will have source network filters so you can restrict from which the requests can be requested.
ALEXANDER AZIMOV: Thank you for your answers.
MASSIMO CANDELA: Thank you very much.
AUDIENCE SPEAKER: Harry Cross here, just an extension of the last point. Have you thought about adding authentication so that you can control a little bit further who can request a reverse?
VALENTIN HEINRICH: No. We have not yet thought about authentication. We wanted to keep it as simple as possible.
MASSIMO CANDELA: Thank you very much.
(Applause)
.
Okay. And then we go with the next presenter, Sander. So, Sander Constantin is a Ph.D. student in Germany, he is research is centered around modern network protocols where he analyses their usage and performance.
SANDER CONSTANTIN: My name is Constantin. I am at university but this is joint work with my colleagues and also colleagues from Technical University of Munich and we looked into using ECN with QUIC. We can also, we looked add in the wide and made a large scale measurement campaign to check where it can be used. Before getting into the datasets, let me give a short recap on ECN and how it can be used. So let's say we have a transport connection and we are not using ECN at the moment and everything is fine, packets are sent from and are received and our receiver just acknowledges this. But we can see that one of the routers already has a filled up Queue and when we get to the point of course when the queue is full packets are lost. Then our ACK stream is interrupted. Our sender has to wait sometime until it triggers a retransmission and it can also reduce a sending rate to release the congestion.
What we do with our router received in the queue is filling up we can motorcar the ticket now, but can still forward t and this mark packet is then received at the receiver and the receiver can acknowledge it but can also send the sender okay there has been tomorrow congestion, please stop sending and then the congestion is relieved. This requires of course direction on the network layers or ‑‑ on our transport layer because our end have to send back this information to the sender.
And network liar we have a 2‑bit EC T. This basically tells routers that ECN can be used. So this is signalled by using EC T 0 or 1 signals. These signals have the same thing at the beginning but now have different meaning. I will get to that in a minute.
And then there is also the congestion experience which is basically used by the routers to mark packets. And of course our transport layer has to mirror this congestion experience back to the sender So it knows it has to adjust the sending rate.
However, ECN has been standardised after TCP and IP have already been standardized. It has to negotiate ECN report. There is work on this that sees the production in 2001 up to now we see a rising support. But this related work also form path impairments. Of course if there is a router on the path which basically just removes the EC T fields or if there is a middle works which changes the meaning of these fields, this can of course inaudible.
With QUIC, stacks should support ECN right away. In the RFC of QUIC is that I say it's a must that every QUIC step has to mirror this ECN back to the sender. This is kind of okay. It's also said in the next sentence that a QUIC step can decide not to do this, although it's a must. So therefore it's a router on this where this sentence should be changed.
But what QUIC does it has some code point counters. It's not mirroring just ECN. Information back to the sender, and the sender can then use PC M documentation. Some text also already support which is reusing these ECT 0 and ECT 1 fuels to distinguish between different traffic. So this is the coin where the... now have a different meaning than before.
But, we also find there is very low ECN usage on UPT port 443.
So, we see that there are new stacks and new completely new protocol. This new valuation method and we see hints for the usage. The question is whether ECN can be used with QUIC. And for this we look at mirror ECN, why they are not doing so and whether ECN valuation always passes.
Gets to our results. What we do is that we visit websites and... we do this for the Alexa umbrella top lists, these are top listed bases on DNS or somehow other computers top list, so for example, the Alexa top list, reused two by information and we also have (inaudible) which we then scan basically for a QUIC end usage. What we can find is that of 2.7 million domains of our domain top lists we see that around one tent supports QUIC. And only 3.3% of these QUIC domains basically also support ECN. We see that 180 million domains were scanned iBGP our measurements and only 17.3 million support QUIC. And of these 5.6% support ECN. So we are mirroring end signals. Given that mirroring should be mandatory we see very low support.
We also looked into the hosts which are involved. But we see also see a higher support in the hosts among the domains on a relative scale. We see that fewer hosts accountable for more domain, which are not supporting ECN. This is potential centralisation and content providers not supporting ECN. We analysed this further and looked into the ASes which are involved and indeed we find that the big hyper‑scalers like the list here are not supporting ECN. But we see that smaller ASes and for example which are then using TCP, for example in the Cloud CDM is using ECN and we also see some tests by Google where sudden ECN has been mirrored.
Okay, we see that only a few domains support it. The question is now why. And this could be either due to stakes that are ignoring ECN or due to the network clear code points. It could be that a stack is not receiving an ECN code points because one the routers on the path is simply removing information.
And for this we are using traceroute box, we already heard about traceroute. Tracebox is a traceroute variant you could say and similar we are using this to trace the paths of all the domains that are supporting ECN and which are not supporting ECN. We found for 97.5% of the domains that it hasn't been any visible clearing of the path.
For 2% we found a visible clearing and for 0.5 percent we didn't trace this.
And for this 2%, there was we saw an I think is he will tier 1 ISP which impacted the 7.6% of these domains. This especially affected smaller hosters especially after we saw route changes in the December 2022. We are doing our measurements here which is using the German research network and there has been certain changes in the begin of December and after that we suddenly saw a rapid decrease in the end usage here.
But what we can also see is the missing support by content providers if they are not routed through this ISP, is not due to clearing. So we even tested whether they support ECN via TCP. Most of them do. So either the QUIC stacks or the indiscovered middleboxes on the path are simple removing ECN.
Now we see that a lot of free text are not supporting it; they are also routers, but the next question is just because ECN is mirrored doesn't mean that it's also used by our transport connections because QUIC requires an ECN validation to use ECN.
Basically the ECN validation checks the first packets have given a time‑out. So whether they are, there has been ECN blackholing for example or routers that see ECN signal drop packets or there is wrong codes sent. So we are signalling ECT 0 at the beginning but the reply is ECT 1 but there was some missing or under counted code point. For example when the path changes, that one router does not support ECN any more but at the beginning it did. And we looked into this. And found that on 0.2% of the come net organise domains passed this validation. A lot of the failed. 96 percent of the mirroring domains that failed, basically failed due to two reasons.
Under counting and remarking. Under counting we found Google AS. We received very weird scenarios where suddenly we see only one packet for example marking ECN, already we are exchanging a lot more packets or we see a lot of marked packets or that we are not marking our packets at the beginning. And related works expects that Google is using some kind of data centres. It could be QUIC is leaks some internal through its connection.
And another way that we found is that ECN is under countable is that light speed server. Which we already heard at the beginning that the small hosters are using and here at the connection initialisation phase, ECN was still in the route but at some point the implementation switches between packet phases and from that point one ECN is disabled. For the remarking we again found network elements of the tier 1 ISP. So there are not just trailing code points but also rewriting code points in the routers and again Google's AS is probably due to the usage.
What does all of this mean put together.
ECN with QUIC is challenged due to multiple effects. We have seen several QUIC stacks do not mirror ECN. We have seen this some network elements clear signals and we have seen often ECN validation fails due to stack and network impairments. What we also saw that was usage is not limited on a global scale. On a global scale we do not see more than 3.4% of ECN validation. Basically with all of the domains, let's not say all of the domains but many many of the domains that are supporting ECN, QUIC, ECN cannot be used. And we also saw that the usage is limited for IPv6 because especially the hyper‑scalers are using IPv6 and the smaller hosts often do not support t so the ECN validation ranges are even lower.
We also saw this on a longitudinal scale, but I will get to that in a moment.
However, this is not just interesting for ECN. But these findings also present challenges for novel ECN mechanisms such as L41. A lot of them are already supporting this, but L4S can also be used with TCP. It's very important that the EC T 0 and 1 signals are not changed. We saw a lot of these ECT 0 to ECT 1 remarkings on our path. And this remarking is detrimental because then traditional L4S is picking up on the same queues. This gets a lot of ECN signals, for example, a lot of congestion experience signals and the traditional traffic ‑‑ ‑‑ here.
But, this is the good thing that we saw in our measurements, the trend is probably to be increased, we see a lot of changes over time and the growing trend. RFC which may trigger a row route of QUIC steps. If we probably contributory negligence err 100 percent IPv4 but we see a lot of changes in stacks and we have a lot of changes in ISP operated backs its routers to see why ECN fields are changed here.
Thank you. (Applause)
NINA BARGISEN: Thank you very much. Do we have questions?
STENOGRAPHER: How can somebody talk so fast?
NINA BARGISEN: I have a question. Do you think that given if the trends continue and we would reach a point where the ISPs will be free of remarking and all the QUIC stacks would vary again, we would never see packet loss on the Internet again?
SANDER CONSTANTIN: That's a good question just because ECN is not impaired on the paths doesn't exactly mean that's working. This is really hard to check because of course this congestion experience signal has to be set at some point so the routers have to decide when to set it and just because they are not impairing the ECN signals doesn't really mean that they are also using these methods to decide when to say, when to mark packets, as congestion experience, so it's hard to say probably a lot of routers are not configured in that way, so I guess packet loss will be here to stay.
NINA BARGISEN: Damn! Okay. Well, thank you very much anyways. It was really interesting.
(Applause)
.
So, next up is Robert from the RIPE NCC. As we all know Robert has been the lead of the RIPE NCC research and development activities for the past 15 years. And is now a principal engineer at the RIPE NCC. So, he is going to do the RIPE NCC tools update, as always. And thank you.
ROBERT KISTELEKI: Thank you very much. 15 minutes. What am I going to do with 15 minutes? I usually get minus 2. That's fine. But we're still here, we are still going to do that.
As usual I am giving you an update about the measurement related activities of the RIPE NCC in this form. First of all, RIPE RIS. So that's the routing nformation system, as you probably all know.
The RIS team plans to make a change in how routing beacons work. This change is scheduled for Q1, 2004. The datasets you can read in a Labs article that we published during the summer time. The short version of that is that some of you may not RIS is not only a passive measurement system, but it also injects some stuff into the routing system, in particular beacons, so there are things that go announced an then revoked later on.
As I said, the team wants to change that behaviour. Particulars are in that article. And of course, this Working Group and also the RIS users will be in the loop of this.
Peering strategy. So, not ‑‑ it's not always true that more is always good, more information is better. And what we have seen is RIS is collecting a lot of redundant data from full feed peers from other peers and that results in increasing volumes of data that we have to store and provide to the community.
So, the question is: Should we change this? And the team has been moving into this direction recently in the last year or two. As you can see, they have been targeting particular populations, tier 1 networks, and then countries that are not covered and then regions. And we wrote up what we have seen as an outcome of that, again on RIPE Labs, so you can go there and read the datasets.
For this reason, for the moment, we temporarily suspended the peering form, so you can't just fill it in and say let's peer together. And if you have stuff to say about this, then Michela is one person we can talk to or Elena, not in this room, she is in a different room, you can also talk to us via the usual channels, forum and the mailing list and so on.
RIPEstat. As you probably know, RIPEstat has what we call I think the team internally called it the classical UI, that was ended in 2013. This is a collection of widgets and some framework around that. You can use your widgets yourself, you can embed it in pages and so on. This has been ageing so the team had to look at it because it was using older technologies so they upgraded it to revitalised it so to speak. Feel free to use it still. And if you are using the old version was widget API, you are strongly encouraged to move on because the new one is better and more secure as well.
At the same time, they have introduced a new UI which we anticipated to be generally better than the old one, but the feedback that we got is not necessarily confirming this. So the team is looking at again whether this is a direction that they should still support, whether the old UI should come back or go, where there are different niches where they are useful. They are reviewing all that feedback and going to make a decision or at least a proposal to go forward in the coming period.
They also brought back the M‑Lab widgets as we call them, these are widgets that are powered by data coming from the measurement lab people, thank you for that. They include the bandwidth capacity. So any random network has bandwidth measurements run by M‑Lab and their partners and also network activities which sections of the network were active and so on.
And these support prefixes and countries as well.
The Stat Team is also focusing on the data quality. Of course you can imagine all the services have ups and downs, hiccups and so on. They are trying to stabilise the backend and I think they made good progress on there. They are also looking at the data quality presented by RIPEstat because that's one of the feedback that we get, that we should really be focusing on that. Not only on stat in general, but in this particular case in at that time.
.
Now RIPE Atlas. Some of you may recall that RIPE Atlas was launched here in Rome, or there in Rome almost exactly 13 years ago. So it was November 1st, I think, or 10th or so, 2010. And Andreas and Victor were sitting in the hall and giving out probes to people who were interested. That's an interesting historical remark.
What's happening nowadays is that we are working on renewing the user interface of RIPE Atlas. You may have seen changes already. In particular ‑‑ sorry, this is this is coming later on ‑‑ renewing the big data back end that we have. We have been using a dual /8 PACE backend for a long time now it's ageing, we have to look at how that works and what can we do with that.
And in general, we are trying to be more cost conscious and looking at options about how we can still store and provide this data.
In particular, we are going to move older data out of what we call this hot store into a colder store and try to do it in such a way that the users don't even notice. So if you are asking for older data you will still get it. You may have to wait a slightly longer period, but you will still get it. That's the plan at least. And at the same time, we are renewing what we call the hot store so that's like the most recent in the last couple of days or weeks or months.
We are also renewing the infrastructure components. You can imagine that there are a whole bunch of components running the show, managing the probes, storing the data and the metadata and so on, serving the data to the users in interface pass and so on. We are busy renewing those as well. For all of these we are looking at how the Cloud technologies can support us, whether it makes sense to move some data or components onto the Cloud when it doesn't make sense.
Now, we come to the user interface, you probably have seen changes, in particular the documentation side and the coverage and statistics have been renewed already, but we are also put out a new measurement for, measurement listing, probe listing, these are meant to be nicer, better, more responsive than the old investigations. And we plan to decommission the old versions after the RIPE meeting. So, they are still there, if for some reason you need to go back to the old forms, the old listings, you can still do that.
And we also put out what we call the beta‑ui, so you really want to look at what is our kind of ideal goal where we want to get, to then feel free to go there and take a peek.
Also, Michela is doing user testing, so usability, user interface, user experience and so on, feel free to find other in the coffee break area. Thank you.
We're still working on the pro firmware packaging as well.
Now, so in the second half of the presentation, I'm going to talk about two proposals that the RIPE Atlas team and the NCC in particular is putting out for the community and the MAT Working Group, because of we see what's happening in the system and we get signals that, you know, some things may need to change. So, we would like to be proactive and make two proposals in particular.
One is, the role of the aggregators. A recent discussion on the mailing list highlighted that there are users of RIPE Atlas who are acting as some kind of an integrators, repackaging Atlas' functionality into their own offerings, and these could be sponsors, they could be profits, non‑profits and so on and so forth. But they do provide additional value to the community via these activities. So, we have to recognise that.
Especially in cases where the SLAs team itself or the service itself cannot get there because of resource reasons or other reasons. But this also skews our understanding of what the users are doing, how many there are, and why they are using the service. So, what the proposal is is to basically put this on the table, so to speak and not just under the table. So let's formally recognise that this thing exists, let's call them aggregators for the sake of argument. We want to make ‑‑ change the framework that we have to accommodate for this. And just say yeah, this is a thing. But then if you do that, then the following rules apply.
For example, we would propose that this should be an account that is identified as such, this is an aggregated account as opposed to ‑‑ I'm using this for the whole of my knock activities. So my knock people are using also this account.
Most importantly, I think the aggregators would need to supply some kind of identification of the users. Now we're not talking about PI, personal information, here, but it is more like a signal that this is client is a different one this than that client.
This would also let the aggregators do kind of anonymous service ifs they want to do that. So we don't necessarily need IP addresses, e‑mail addresses, anything that the aggregators may not even know but it is a signal that they are different clients and how many of them.
As you can imagine, at some point if this goes above some limit, then, you know, this is a lot of usage. So, if an aggregator is doing 100 measurements a day or 1,000 measurements a day, that's probably fine. But when they do multiple thousands or millions of measurements in a day in the extreme case that is something that we might want to emphasise that then please, please, please be a sponsor of RIPE Atlas.
Okay. So with this change, we expect that the community will be aware of this, that this is actually happening, and then we can consistently enforce the rules against it, right, so if there are responsibilities and benefits, then all of those would be consistently applied to everyone.
And internally and when we report back to you about usage and so on, this would let us count properly to know how many users we actually have, because if you think about it in the extreme case, if there are ten aggregators and everyone is using the aggregator services we would think that we have ten users. And that's not correct.
So, then the other proposal. RIS and Atlas have been around for some years now, since 1999 and 2010. And they have been collecting and serving that data to the community and to all of the users. All the historical data is still available that we have collected. There is arguably high value for this data. We sometimes here that explicitly set out. Sometimes we understand it from the actions of people that this data is still used, even if it's historical, even if it's old, there is a lot of value to this. But, for the NCC purposes, it has costs that we have, we still keep around this data, we serve it to you if you ask for it. That definitely comes with some costs.
In our efforts to be a bit more cost conscious, we are reviewing if we need to do this, if we need to keep on doing this, or if we need to change.
So, in particular, we put out a proposal and both of the proposals, by the way, have labs articles behind them, so if you want to read the full text, I would ask you to read those articles. But in this case about the data retention, we would like to propose a principle, so to speak. We don't want to call it poll but let's call it principles.
First of all, this is mostly in from more generic to more specific order. So first of all, the NCC should retain as much data as possible in a financially sustainable way. Right. So, that's literally what it is.
In general, all this data should be able to the users. Right, if we keep it, we don't want to keep it to ourselves and do analysis that you yourself couldn't do. The data should be available to everyone. However, because some data is accessed more frequently, especially the newer data is accessed, as it ages it's less and less frequently used we want to tailor the system such that we find cheaper solutions for data that is not as hot, not as frequently used. It's logical, but it's one of the principles that we want to lay down.
Then, of course Atlas is different than RIS. They probably will have different systems and different cutoffs and different policies on what we call old data, what we call new data. So these kind of tiers would be tailored to the system itself.
We would also like to ask people to identify themselves when they are asking for this data. At the moment, fully anonymous access to all this data is possible, which is okay, that's totally fine. But then if we, in cases exceed bandwidth limits, so on and so forth, it is highly useful to know who these people are if they can pull them together or we need to provide bigger capacity to tomorrow group of users. It would also let us know how many people are actually downloading the data and maybe we can ask why that is.
The most specific thing is RIPE Atlas has something that we call non‑public measurements, you can call them private measurements but the official term is non‑public measurements. And there are reasons for this. The users have been asking for this. So that's fine. But this data is not available by its virtue, it's private data, it's non‑public measurement, it's not available to everyone. So we would like to propose that the retention period for this should be much, much, much shorter because it is only really useful for the people who wanted it and for them probably only within the short‑term.
So these are two proposals, and I guess it's going to be up to the Working Group Chairs to decide how the discussion is happening. But from our perspective, we would really love to hear you if you agree, disagree, should we change these? In general it's fine, just go for it, or not? Because this is, from our perspective, a proposal.
And while the discussion is happening, I have been asked by the lovely training team to put up this slide and if you want to have some feedback to them you can do that right now or later as well.
STEPHEN STROWES: All right, thank you, Robert. We have in the queue, Simon is in the queue on Meetecho:
AUDIENCE SPEAKER: Thanks for working on this. I am happy to ‑‑ that you consider the needs of these aggregators, and actually this is something I'd like to encourage you to do and try not to make the requirements too onerous. I see the NCC as the stewards of the Atlas data, and hope that you will not fall in the trap of thinking you have to ‑‑ a kind of a monopoly and have to see how to share the data and lose control of like accurately counting the visitors. I hope this can be done in a pragmatic way.
On the retention, I think that's also good thinking. What I'm wondering about is this goal of sustainability, how can you decide, because I see that this can, like, easily be skewed one way or the other, how do you decide whether the costs of storing the data, or retaining the data is sustainable compared to what? Is that like income related to this data or are you looking for sponsors or something to ensure sustainability? That part is unclear to me. So me it's a very qualitative, someone, some manager will decide this is too expensive and they will say it's not sustainable and then it will be thrown away. Or do you have any objective criteria in mind?
ROBERT KISTELEKI: Not at the moment. I think we are also looking for guidance on that, what can be considered sustainable. But let me give you a negative answer by what is sustainable is to say please keep all the data and don't spend money on it. So that would be somewhat unsustainable. So, of course, we want to, and can be more specific than this. But that's, you know, as a principle, as a guiding line, this is what we want on the highest level I think that needed to be said.
STEPHEN STROWES: Okay. We are about to run a little bit over time. We are going to allow that. We are going to give a few more minutes for this because this is an interesting topic but if we can keep questions fairly short. This is going to be taken to the mailing list for sure.
AUDIENCE SPEAKER: Gert Döring, I generally agree to the principles proposed. On the sustainability bit, it might be useful to actually have numbers, like we are participating 50 million for the storage of the data every month, and if we just move it all to this drive it will be down to €10. More realistic things like. I could see like fast access in the Cloud being really, really expensive and having it somewhere being less expensive but I don't know how much. And of course this needs to be in relation to the budget and the NCC budget discussions and all of that. But I actually wanted to ask about the aggregators.
If they can do thousands of measurements a day, they have to have credits. So, if they have credits, they provide probes. So, they bring value. So, having the aggregators here is a good thing, because they bring measurement value. But trying to formalise this in a useful way is also a good thing. So go on!.
ROBERT KISTELEKI: Okay. Thank you.
AUDIENCE SPEAKER: Randy Bush, IIJ, research and Arrcus. I am a consumer, and I am not going to weigh in on these proposals. I am interested still for the 1,312th time on knowing what kind of peering each RIS peer gives. Is it a customer view, an internal view etc., etc.? We have discussed this just ‑‑ I'd like to keep that question alive. But also love the security test of the QR code.
ROBERT KISTELEKI: Thank you, Randy.
AUDIENCE SPEAKER: Massimo Candela. The first thing is as a MAT Working Group Chair is like, if you can briefly provide exactly what the people have to do to ‑‑ like do you have a form, do you have a questionnaire? Do you have something? Do they have to send just an e‑mail about the two proposals? That's the first question.
The second question is, I removed the hat of MAT Working Group Chair. So, I think the entire topic ‑‑ okay, I'll just say that I am strongly against the aggregator thing, I'm sorry. I think the entire topic started because of lack of attribution and I think we should focus on that. And the aggregator thing, the original deal was you have credits because you host a probe with the credits you do whatever you want. I am totally in with providing with you metadata so that you can recognise user, whatever. But I am against, like, for example, things like oh, if you reach a certain level, then you have to sponsor, something like this. So if addition to host probes you have to sponsor basically, to me it looks a bit like commercialising. There is a free tier and then you have to sponsor.
I mean, I really prefer the model that you host, you have the credits, whatever you want to do with it, that's fine. Also, we all do some kind of earning with Atlas in a way. If I do troubleshooting for work, I am doing something that in the end it is business related. So, I really don't see how you can draw the line, and it looks to me something that is going to complicate and put limits on how to us auto the data and the usefulness of hosting probes and stuff. So this is my feedback, as I said, Not as a Chair. Please.
ROBERT KISTELEKI: Okay, so answer your first one. We have a forum topic open for one proposal and another forum topic for the other. We have the MAT Working Group mailing list. I personally, you know, encourage everyone, whichever you prefer, to provide feedback there. But it's not only feedback. I think it's a discussion that we need to have. Like what you just said, makes sense to me. This is one view. Whether, you know, that is a consensus or not, that we don't know yet, but that's why we would like to hear people's opinions and input to this.
AUDIENCE SPEAKER: Chris a. I am a big user of RIS and RIPE Atlas data. Which I find very useful, and I think storing the ‑‑ all this data on storage makes perfectly sense. What we have been working on with some of my Ph.D students in post docs is on looking forward somehow to store the data, because, as you said, there is a lot of redundancy, so we are investigating ways to determine what is redundant right from the start and to discard it right away. So we have a submission that op nets was presented today, so you may want to have a look and we will collaborate on this if you are interested.
ROBERT KISTELEKI: That's a really good idea. Thank you.
AUDIENCE SPEAKER: Just a quick comment. Mime Ivan Beveridge. As you are aware I believe that some of the use of the aggregators is potentially commercial, but some is kind of providing facility for other users, so, you know, this person may be doing it, kind of for free for hosting a website and providing functionality. So it's not necessarily that there is commercial gain from that. It's just a comment rather than anything.
ROBERT KISTELEKI: That's very fair and I can imagine a scenario where we make a distinction between academic aggregation versus for‑profit versus non‑profit. You know, that's I guess it's all possible. Whether we want to go that far, that's ‑‑ I guess that's part of the discussion.
STEPHEN STROWES: Thank you Robert. Thank you all.
There are topics on the mailing lists. We will probably bump those. It would be good to continue the conversation over there.
And so this was our agenda. So this is the ‑‑ this was our agenda. Thank you to all of our speakers. And yes, so comments and remarks on to the lists, either directly to us for the Chairs or onto the general MAT Working Group mailing list.
Two important announcements that we need to bring up. One is verbal. The PC elections are open. So vote early but hopefully don't vote often. You can find the nominee bios on the RIPE 87 website. More important logistics for this evening, there will be shuttle buses running to the boat, I hear we're going to be on a boat and the buses will be departing from buildings 1 and 2, the return buses are returning every 30 minutes, but apparently there is only one return bus that is return to the IBIS hotel. So if you are staying there you will have to enjoy yourself later. That's a requirement. This is mandatory fun.
That's it. See you at RIPE 88. Thank you all.
(Applause)
LIVE CAPTIONING BY
MARY McKEON, RMR, CRR, CBC
DUBLIN, IRELAND.
