We are in the process of migrating this forum. A new space will be available soon. We are sorry for the inconvenience.

Routing Hansenet <-> OVH is completely broken


schwarzlicht
19.11.08, 21:48
Hmm, ist schon etwas komisch, die von OVH beeinflussbare Route sieht normal aus:
Code:
x2:~# traceroute 213.191.64.100  --->>>  hope2 von D@ni3l
traceroute to 213.191.64.100 (213.191.64.100), 30 hops max, 40 byte packets
 1  rbx-20-m2.routers.ovh.net (91.121.75.252)  0.608 ms  0.412 ms  0.542 ms
 2  rbx-1-6k.routers.ovh.net (213.251.191.1)  3.028 ms *  0.882 ms
 3  40g.rbx-2-6k.routers.ovh.net (213.251.128.18)  4.561 ms * *
 4  160g.th2-1-6k.routers.ovh.net (213.186.32.202)  5.893 ms * *
 5  * 20g.fra-1-6k.routers.ovh.net (213.251.130.14)  13.790 ms *
 6  decix2.hansenet.net (80.81.193.232)  15.013 ms  13.773 ms  13.966 ms
 7  ae1-102.cr01.fra.de.hansenet.net (62.109.109.253)  14.754 ms  14.710 ms  15.137 ms
 8  so-1-0-0-0.cr01.dus.de.hansenet.net (213.191.66.29)  16.783 ms  17.558 ms  16.896 ms
 9  lo1.br10.aah.de.hansenet.net (213.191.64.100)  19.246 ms  20.368 ms  20.594 ms
Liegt es an Hansenet?

ALiEn
19.11.08, 21:42
Kann ich bestätigen. Heute morgen hatte ich Probleme mein Server zu erreichen. Konnte aber keine weiteren Tests durchführen.

Ok von 1und1 hab ich derzeit nur max. 500kb/sek. Wenn ich das richtig sehe ist aber auch th2-1-6k zu Alice und fra-1-6k stark ausgelastet.

D@ni3l
19.11.08, 20:49
hi oles,

the routing is broken once more.

A ping -c 100 had 3% loss

Ping-Statistik für 91.121.149.61:
Pakete: Gesendet = 100, Empfangen = 97, Verloren = 3 (3% Verlust),
Ca. Zeitangaben in Millisek.: Minimum = 36ms, Maximum = 1646ms, Mittelwert = 157ms
here some tracerouts

Routenverfolgung zu daniel-schmitz.eu [91.121.149.61] über maximal 30 Abschnitte
:

1 2 ms 2 ms 2 ms SL2141I.thisisonlymynet2 [192.168.1.1]
2 35 ms 79 ms 65 ms lo1.br10.aah.de.hansenet.net [213.191.64.100]
3 20 ms 20 ms 20 ms ae1-102.cr02.dus.de.hansenet.net [62.109.110.126]
4 121 ms 23 ms 23 ms so-6-0-0-0.cr02.fra.de.hansenet.net [213.191.87.221]
5 22 ms 23 ms 24 ms gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.178]
6 292 ms 249 ms 25 ms decix.routers.ovh.net [80.81.192.209]
7 * * * Zeitüberschreitung der Anforderung.
8 108 ms * 227 ms 160g.rbx-2-6k.routers.ovh.net [213.186.32.201]
9 38 ms 37 ms 99 ms rbx-47-m1.routers.ovh.net [91.121.130.141]
10 263 ms 142 ms 62 ms daniel-schmitz.eu [91.121.149.61]

Ablaufverfolgung beendet.
Routenverfolgung zu daniel-schmitz.eu [91.121.149.61] über maximal 30 Abschnitte
:

1 2 ms 2 ms 2 ms SL2141I.thisisonlymynet2 [192.168.1.1]
2 35 ms 79 ms 65 ms lo1.br10.aah.de.hansenet.net [213.191.64.100]
3 20 ms 20 ms 20 ms ae1-102.cr02.dus.de.hansenet.net [62.109.110.126]
4 121 ms 23 ms 23 ms so-6-0-0-0.cr02.fra.de.hansenet.net [213.191.87.221]
5 22 ms 23 ms 24 ms gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.178]
6 292 ms 249 ms 25 ms decix.routers.ovh.net [80.81.192.209]
7 * * * Zeitüberschreitung der Anforderung.
8 108 ms * 227 ms 160g.rbx-2-6k.routers.ovh.net [213.186.32.201]
9 38 ms 37 ms 99 ms rbx-47-m1.routers.ovh.net [91.121.130.141]
10 263 ms 142 ms 62 ms daniel-schmitz.eu [91.121.149.61]

Ablaufverfolgung beendet.
and a last one

Routenverfolgung zu daniel-schmitz.eu [91.121.149.61] über maximal 30 Abschnitte
:

1 2 ms 2 ms 2 ms SL2141I.thisisonlymynet2 [192.168.1.1]
2 35 ms 79 ms 65 ms lo1.br10.aah.de.hansenet.net [213.191.64.100]
3 20 ms 20 ms 20 ms ae1-102.cr02.dus.de.hansenet.net [62.109.110.126]
4 121 ms 23 ms 23 ms so-6-0-0-0.cr02.fra.de.hansenet.net [213.191.87.221]
5 22 ms 23 ms 24 ms gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.178]
6 292 ms 249 ms 25 ms decix.routers.ovh.net [80.81.192.209]
7 * * * Zeitüberschreitung der Anforderung.
8 108 ms * 227 ms 160g.rbx-2-6k.routers.ovh.net [213.186.32.201]
9 38 ms 37 ms 99 ms rbx-47-m1.routers.ovh.net [91.121.130.141]
10 263 ms 142 ms 62 ms daniel-schmitz.eu [91.121.149.61]

Ablaufverfolgung beendet.
it seems to be in the decix, as the last time.
Maybe you can can ask there once more, this work much better than my mails to the hansenet service that thier routing is broken in decix

rebrain
10.11.08, 22:41
1 34 ms 52 ms 34 ms lo1.br11.ber.de.hansenet.net [213.191.64.20]
2 32 ms 33 ms 33 ms ae0-101.cr01.ber.de.hansenet.net [62.109.108.61]

3 51 ms 51 ms 52 ms so-6-0-0-0.cr01.fra.de.hansenet.net [213.191.87.
242]
4 52 ms 51 ms 51 ms gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.1
78]
5 * * 53 ms decix.routers.ovh.net [80.81.192.209]
6 * * * Request timed out.
7 * 64 ms * 160g.rbx-2-6k.routers.ovh.net [213.186.32.201]
8 65 ms 66 ms 64 ms rbx-15-m1.routers.ovh.net [213.251.191.168]
9 65 ms 66 ms 63 ms epic-zone.com [91.121.31.116]

Trace complete.
The ping is a bit bigger than it was before, but at least it is not over the rooftops.

oles@ovh.net
08.11.08, 23:30
bjo a écrit:
>
> works for me, too:


great ! thanks for feedback !


bjo
08.11.08, 19:26
works for me, too:

bjo@majestix:~$ traceroute nord-west.net
traceroute to nord-west.net (91.121.172.75), 30 hops max, 40 byte packets
1 routenix (192.168.0.1) 0.832 ms 0.925 ms 1.040 ms
2 lo1.br50.asham.de.hansenet.net (213.191.84.234) 33.047 ms 33.470 ms 34.393 ms
3 ae1-104.cr01.weham.de.hansenet.net (62.109.119.125) 34.900 ms 35.347 ms 36.029 ms
4 so-7-0-0-0.cr01.dus.de.hansenet.net (213.191.87.250) 42.436 ms 43.358 ms 44.305 ms
5 so-3-0-0-0.cr01.fra.de.hansenet.net (213.191.87.245) 61.047 ms 61.202 ms 61.465 ms
6 gi5-0-0.pr01.decix.de.hansenet.net (62.109.109.179) 51.109 ms 51.844 ms 52.482 ms
7 * decix.routers.ovh.net (80.81.192.209) 42.194 ms *
8 * * *
9 160g.rbx-2-6k.routers.ovh.net (213.186.32.201) 57.157 ms * *
10 rbx-53-m1.routers.ovh.net (91.121.130.159) 60.263 ms 60.358 ms 61.930 ms
11 asterix.nord-west.net (91.121.172.75) 64.350 ms 65.036 ms 65.487 ms

Kademlia
08.11.08, 19:22
Working for me. Thanks had the same problems for about 3 or 4 days.

oles@ovh.net
08.11.08, 16:50
bjo a écrit:
>
> still exists:


thansk for feedback. I've asked decix's noc to fix the problem.


schwarzlicht
08.11.08, 15:00
Code:
7  bzrb403.equant.net (80.81.192.148)  296.928 ms  280.131 ms  273.196 ms
Das ist Orange (FranceTelecom), der Router scheint überlastet:
Code:
 traceroute 80.81.192.148
traceroute to 80.81.192.148 (80.81.192.148), 30 hops max, 52 byte packets
 1  91-143-95-6.vserver.kundencontroller.de (91.143.95.6)  0.069 ms  0.040 ms  0.021 ms
 2  bzrb403.equant.net (80.81.192.148)  274.516 ms *  276.878 ms

ping 80.81.192.148
PING 80.81.192.148 (80.81.192.148) 56(84) bytes of data.
64 bytes from 80.81.192.148: icmp_seq=1 ttl=243 time=248 ms
64 bytes from 80.81.192.148: icmp_seq=2 ttl=243 time=246 ms
64 bytes from 80.81.192.148: icmp_seq=3 ttl=243 time=245 ms
64 bytes from 80.81.192.148: icmp_seq=4 ttl=243 time=242 ms

rebrain
08.11.08, 14:11
It seems to be a problem with this one:
5 315 ms 310 ms 313 ms bzrb403.equant.net [80.81.192.148]
Still having a problem with the ping.

EDIT: Just seen the link in the message before. Hopefully they will solve it soon.

bjo
08.11.08, 14:06
still exists:
traceroute to nord-west.net (91.121.172.75), 30 hops max, 40 byte packets
1 routenix (192.168.0.1) 0.865 ms 0.942 ms 1.062 ms
2 lo1.br50.asham.de.hansenet.net (213.191.84.234) 26.309 ms 26.416 ms 27.684 ms
3 ae1-104.cr01.weham.de.hansenet.net (62.109.119.125) 28.359 ms 29.314 ms 30.524 ms
4 so-7-0-0-0.cr01.dus.de.hansenet.net (213.191.87.250) 38.475 ms 39.375 ms 40.073 ms
5 so-1-1-0-0.cr01.fra.de.hansenet.net (213.191.66.30) 44.495 ms so-3-1-0-0.cr01.fra.de.hansenet.net (213.191.87.158) 45.499 ms so-3-0-0-0.cr01.fra.de.hansenet.net (213.191.87.245) 46.955 ms
6 gi5-0-0.pr02.decix.de.hansenet.net (62.109.109.243) 47.348 ms 47.574 ms 48.696 ms
7 bzrb403.equant.net (80.81.192.148) 296.928 ms 280.131 ms 273.196 ms
8 so-7-2-0-0.fftcr2.Frankfurt.opentransit.net (193.251.248.209) 274.829 ms 275.523 ms 276.587 ms
9 so-1-2-0-0.fftcr4.Frankfurt.opentransit.net (193.251.242.198) 277.604 ms 278.550 ms 279.032 ms
10 xe-4-2-0.edge3.frankfurt1.level3.net (4.68.127.197) 290.342 ms 291.278 ms 292.244 ms
11 ae-31-53.ebr1.Frankfurt1.Level3.net (4.68.118.94) 304.557 ms ae-32-54.ebr2.Frankfurt1.Level3.net (4.68.118.126) 298.365 ms *
12 ae-1-100.ebr2.Frankfurt1.Level3.net (4.69.132.126) 305.693 ms ae-2.ebr1.Dusseldorf1.Level3.net (4.69.132.137) 294.288 ms 294.276 ms
13 ae-1-100.ebr2.Dusseldorf1.Level3.net (4.69.132.130) 304.215 ms 299.729 ms ae-2.ebr1.Dusseldorf1.Level3.net (4.69.132.137) 304.668 ms
14 ae-2.ebr1.Amsterdam1.Level3.net (4.69.133.89) 301.956 ms ae-1-100.ebr2.Dusseldorf1.Level3.net (4.69.132.130) 316.220 ms 316.462 ms
15 ae-1-53.edge3.Amsterdam1.Level3.net (4.68.120.82) 310.336 ms ae-2.ebr1.Amsterdam1.Level3.net (4.69.133.89) 316.802 ms ae-1-55.edge3.Amsterdam1.Level3.net (4.68.120.146) 312.770 ms
16 ae-1-53.edge3.Amsterdam1.Level3.net (4.68.120.82) 314.336 ms ae-1-55.edge3.Amsterdam1.Level3.net (4.68.120.146) 316.443 ms 316.809 ms
17 30g.level3.ams-1-6k.routers.ovh.net (213.251.130.69) 315.476 ms * *
18 rbx-53-m1.routers.ovh.net (91.121.130.159) 318.872 ms * *
19 asterix.nord-west.net (91.121.172.75) 306.724 ms rbx-53-m1.routers.ovh.net (91.121.130.159) 305.079 ms asterix.nord-west.net (91.121.172.75) 305.804 ms

oles@ovh.net
07.11.08, 23:40
oles@ovh.net a écrit:

probably resolved

http://travaux.ovh.com/?do=details&id=2590


oles@ovh.net
07.11.08, 23:08
rebrain a écrit:
>
> Here is my traceroute:


still the problem with ?


rebrain
07.11.08, 22:17
Here is my traceroute:

Tracing route to epic-zone.com [91.121.31.116]
over a maximum of 30 hops:

1 100 ms 32 ms 33 ms lo1.br11.ber.de.hansenet.net [213.191.64.20]
2 43 ms 32 ms 35 ms ae1-102.cr01.ber.de.hansenet.net [62.109.108.125
]
3 49 ms 51 ms 51 ms so-1-0-0-0.cr01.fra.de.hansenet.net [213.191.66.
21]
4 51 ms 61 ms 52 ms gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.1
78]
5 315 ms 310 ms 313 ms bzrb403.equant.net [80.81.192.148]
6 320 ms 317 ms 326 ms so-7-2-0-0.fftcr2.Frankfurt.opentransit.net [193
.251.248.209]
7 332 ms 335 ms 331 ms so-1-2-0-0.fftcr4.Frankfurt.opentransit.net [193
.251.242.198]
8 356 ms 349 ms 347 ms xe-4-2-0.edge3.frankfurt1.level3.net [4.68.127.1
97]
9 * 327 ms * ae-31-51.ebr1.Frankfurt1.Level3.net [4.68.118.30
]
10 * 359 ms 359 ms ae-1-100.ebr2.Frankfurt1.Level3.net [4.69.132.12
6]
11 326 ms 326 ms 323 ms ae-2.ebr1.Dusseldorf1.Level3.net [4.69.132.137]

12 319 ms 326 ms 324 ms ae-1-100.ebr2.Dusseldorf1.Level3.net [4.69.132.1
30]
13 339 ms 331 ms 337 ms ae-2.ebr1.Amsterdam1.Level3.net [4.69.133.89]
14 327 ms 335 ms 343 ms ae-1-55.edge3.Amsterdam1.Level3.net [4.68.120.14
6]
15 361 ms 366 ms 433 ms 30g.level3.ams-1-6k.routers.ovh.net [213.251.130
.69]
16 338 ms 330 ms * 40g.rbx-2-6k.routers.ovh.net [91.121.131.9]
17 320 ms 324 ms 318 ms rbx-15-m1.routers.ovh.net [213.251.191.168]
18 339 ms 375 ms 337 ms epic-zone.com [91.121.31.116]

Trace complete.

easynetwork
07.11.08, 19:26
Code:
  1     1 ms    <1 ms    <1 ms  192.168.1.1
  2    63 ms    48 ms    48 ms  lo1.br02.asham.de.hansenet.net [213.191.84.201]

  3   219 ms   217 ms    48 ms  ae1-104.cr01.weham.de.hansenet.net [62.109.119.1
25]
  4    84 ms    72 ms   186 ms  so-7-1-0-0.cr01.dus.de.hansenet.net [213.191.87.
145]
  5   150 ms    57 ms    57 ms  so-3-1-0-0.cr01.fra.de.hansenet.net [213.191.87.
158]
  6    56 ms    66 ms    56 ms  gi5-0-0.pr02.decix.de.hansenet.net [62.109.109.2
43]
  7   286 ms   289 ms   293 ms  bzrb403.equant.net [80.81.192.148]
  8   295 ms   291 ms   288 ms  so-7-2-0-0.fftcr2.Frankfurt.opentransit.net [193
.251.248.209]
  9   294 ms   292 ms   292 ms  so-1-2-0-0.fftcr4.Frankfurt.opentransit.net [193
.251.242.198]
 10   291 ms   288 ms   292 ms  xe-4-2-0.edge3.frankfurt1.level3.net [4.68.127.1
97]
 11   336 ms   282 ms     *     ae-32-54.ebr2.Frankfurt1.Level3.net [4.68.118.12
6]
 12   284 ms   309 ms   342 ms  ae-2.ebr1.Dusseldorf1.Level3.net [4.69.132.137]

 13   292 ms   273 ms   285 ms  ae-1-100.ebr2.Dusseldorf1.Level3.net [4.69.132.1
30]
 14   317 ms   331 ms   284 ms  ae-2.ebr1.Amsterdam1.Level3.net [4.69.133.89]
 15   280 ms   283 ms   281 ms  ae-1-53.edge3.Amsterdam1.Level3.net [4.68.120.82
]
 16   282 ms   288 ms   278 ms  30g.level3.ams-1-6k.routers.ovh.net [213.251.130
.69]
 17   282 ms     *        *     40g.rbx-2-6k.routers.ovh.net [91.121.131.9]
 18   290 ms   293 ms   284 ms  rbx-6-m1.routers.ovh.net [213.251.191.141]
 19   278 ms   275 ms   288 ms  ks3****0.kimsufi.com [87.98.***.***]

Ablaufverfolgung beendet.



Routenverfolgung

  1     1 ms    <1 ms    <1 ms  192.168.1.1
  2    83 ms    54 ms    49 ms  lo1.br02.asham.de.hansenet.net [213.191.84.201]

  3    54 ms    48 ms    47 ms  ae1-104.cr01.weham.de.hansenet.net [62.109.119.1
25]
  4    55 ms    55 ms    53 ms  so-7-1-0-0.cr01.dus.de.hansenet.net [213.191.87.
145]
  5   102 ms    81 ms   186 ms  so-3-0-0-0.cr01.fra.de.hansenet.net [213.191.87.
245]
  6   329 ms   340 ms   309 ms  gi4-0-0.pr02.decix.de.hansenet.net [62.109.109.1
78]
  7   299 ms   335 ms   409 ms  bzrb403.equant.net [80.81.192.148]
  8   320 ms   324 ms   311 ms  so-7-2-0-0.fftcr2.Frankfurt.opentransit.net [193
.251.248.209]
  9   407 ms   321 ms   279 ms  so-1-2-0-0.fftcr4.Frankfurt.opentransit.net [193
.251.242.198]
 10   294 ms   291 ms   292 ms  xe-4-2-0.edge3.frankfurt1.level3.net [4.68.127.1
97]
 11     *      292 ms   290 ms  ae-32-54.ebr2.Frankfurt1.Level3.net [4.68.118.12
6]
 12   316 ms   328 ms   315 ms  ae-2.ebr1.Dusseldorf1.Level3.net [4.69.132.137]

 13   296 ms   306 ms   302 ms  ae-1-100.ebr2.Dusseldorf1.Level3.net [4.69.132.1
30]
 14   306 ms   323 ms   324 ms  ae-2.ebr1.Amsterdam1.Level3.net [4.69.133.89]
 15   470 ms   310 ms   305 ms  ae-1-53.edge3.Amsterdam1.Level3.net [4.68.120.82
]
 16     *        *      302 ms  30g.level3.ams-1-6k.routers.ovh.net [213.251.130
.69]
 17   326 ms   318 ms   319 ms  40g.rbx-2-6k.routers.ovh.net [91.121.131.9]
 18   307 ms   308 ms   317 ms  rbx-6-m1.routers.ovh.net [213.251.191.141]
 19   315 ms   310 ms   309 ms  ks3****0.kimsufi.com [87.98.***.***]

Ablaufverfolgung beendet.
whats going on? normaly i reach my server over decix <-> ovh and not over amsterdam!

any problems in the infrastruckture of the backbones?

here http://weathermap.ovh.net/ i can see that the second backbone to decix is full. any problems with the dataconnection my webserver is realy slow.

oles@ovh.net
07.11.08, 19:04
rebrain a écrit:
>
> The connection to ovh just went down for a couple minutes. To the
> ovh.com website and to any of the servers.
>
> Now it is back and the pings are back to normal!
>
> Thank you very much!


can you send the traceroute ? I have no feedback from Hansenet at the moment.


rebrain
07.11.08, 17:27
The connection to ovh just went down for a couple minutes. To the ovh.com website and to any of the servers.

Now it is back and the pings are back to normal!

Thank you very much!


EDIT: Aaaand we are back to the 250ms.

oles@ovh.net
07.11.08, 17:20
rebrain a écrit:
>
> yeah i got the problem aswell, i am from Germany and my pings are around
> 300 instead of 50. When could this be fixed?


It's Hansenet's side problem. I've contacted them.


rebrain
07.11.08, 16:58
yeah i got the problem aswell, i am from Germany and my pings are around 300 instead of 50. When could this be fixed?

oles@ovh.net
07.11.08, 16:08
>> 5 23 ms 21 ms 21 ms gi5-0-0.pr02.decix.de.hansenet.net
>> [62.109.109.243]
>> 6 590 ms 605 ms 613 ms bzrb403.equant.net [80.81.192.148]
>> 7 663 ms 613 ms * decix.routers.ovh.net [80.81.192.209]


I check it with hansenet.


D@ni3l
06.11.08, 19:50
Hi,

the routing between Hansenet und OVH is broken in decix.

This time it is not ovh's fault:

Routenverfolgung zu daniel-schmitz.eu [91.121.149.61] über maximal 30 Abschnitte
:

1 3 ms 1 ms 1 ms SL2141I.thisisonlymynet2 [192.168.1.1]
2 19 ms 17 ms 17 ms lo1.br10.aah.de.hansenet.net [213.191.64.100]
3 21 ms 19 ms 18 ms ae1-102.cr02.dus.de.hansenet.net [62.109.110.126]
4 27 ms 22 ms 21 ms so-6-0-0-0.cr02.fra.de.hansenet.net [213.191.87.221]
5 23 ms 21 ms 21 ms gi5-0-0.pr02.decix.de.hansenet.net [62.109.109.243]
6 590 ms 605 ms 613 ms bzrb403.equant.net [80.81.192.148]
7 663 ms 613 ms * decix.routers.ovh.net [80.81.192.209]
8 607 ms 583 ms * 20g.th2-1-6k.routers.ovh.net [213.251.130.13]
9 613 ms * 702 ms 160g.rbx-2-6k.routers.ovh.net [213.186.32.201]
10 622 ms 613 ms 614 ms rbx-47-m1.routers.ovh.net [91.121.130.141]
11 581 ms 589 ms 613 ms daniel-schmitz.eu [91.121.149.61]

Ablaufverfolgung beendet.
I know that the problems are in hansenet, because with acror i get ping < 20ms, from t-online ~60ms and wih the dfn network also <20ms.

I already open a ticket with hansenet, but maybe is works much better if you also complain about that.

here is als a trecroute from the server to my home ip:

ircuser@daniel-schmitz:~$ traceroute nslu.daniel-schmitz.eu
traceroute to nslu.daniel-schmitz.eu (78.50.95.36), 30 hops max, 40 byte packets
1 rbx-47-m1.routers.ovh.net (91.121.149.253) 0.549 ms 0.414 ms 0.370 ms
2 91.121.130.130 (91.121.130.130) 0.658 ms * 0.318 ms
3 160g.th2-1-6k.routers.ovh.net (213.186.32.202) 63.610 ms 6.572 ms *
4 20g.fra-1-6k.routers.ovh.net (213.251.130.14) 12.870 ms * 12.861 ms
5 decix2.hansenet.net (80.81.193.232) 15.514 ms 25.081 ms 14.392 ms
6 ae0-101.cr01.fra.de.hansenet.net (62.109.109.189) 14.118 ms 14.135 ms 14.534 ms
7 so-6-1-0-0.cr01.dus.de.hansenet.net (213.191.87.157) 561.761 ms 563.782 ms 556.808 ms
8 * * *
9 * * *
10 * * *
thanks, daniel