Quad Ethernet bug on sparc64

From: Doug Winter <dwinter(at)icpeurope.net>
Date: Tue Jun 10 2003 - 05:47:28 EDT

[ I submitted this yesterday from sendbug, but I haven't seen it come through, so I've forwarded again manually. Apologies if it's a dup. ]

>Submitter-Id: net
net
>Confidential: no
>Synopsis: Quad Ethernet bug on sparc64
>Severity: critical
        

	System      : OpenBSD 3.3
	Architecture: OpenBSD.sparc64
	Machine     : sparc64

>Description:

I had this problem on a new system that's now in a rack, so I've set up a duplicate system on the bench for testing (I suspected a possible hardware problem with the QFE card). The duplicate system has shown the same symptoms.

On a freshly installed OpenBSD 3.3/sparc64 system on a Sun Ultra 5 with a sun Quad Fast Ethernet card.

hme1 (port 0 on the QFE) is connected to a switch, and thence to our office network.

hme2 (port 1) is connected to a seperate, dedicated switch, with a single other machine on it.

tern# ifconfig hme1
hme1: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500

        address: 08:00:20:c8:50:01
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 172.16.1.242 netmask 0xffff0000 broadcast 172.16.255.255
        inet6 fe80::a00:20ff:fec8:5001%hme1 prefixlen 64 scopeid 0x2

tern# ifconfig hme2
hme2: flags=8822<BROADCAST,NOTRAILERS,SIMPLEX,MULTICAST> mtu 1500

        address: 08:00:20:c8:50:02
        media: Ethernet autoselect

tern# ping -c5 172.16.1.2

PING 172.16.1.2 (172.16.1.2): 56 data bytes
64 bytes from 172.16.1.2: icmp_seq=0 ttl=128 time=0.868 ms
64 bytes from 172.16.1.2: icmp_seq=1 ttl=128 time=0.423 ms
64 bytes from 172.16.1.2: icmp_seq=2 ttl=128 time=0.420 ms
64 bytes from 172.16.1.2: icmp_seq=3 ttl=128 time=0.408 ms
64 bytes from 172.16.1.2: icmp_seq=4 ttl=128 time=0.440 ms

--- 172.16.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/std-dev = 0.408/0.511/0.868/0.180 ms

ok, hme1 is working fine...

tern# ifconfig hme2 192.168.5.2
interrupt_vector: spurious vector 7d2 at pil 1 tern# ifconfig hme2
hme2: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500

        address: 08:00:20:c8:50:02
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 192.168.5.2 netmask 0xffffff00 broadcast 192.168.5.255
        inet6 fe80::a00:20ff:fec8:5002%hme2 prefixlen 64 scopeid 0x3

it's up, but i don't like that "spurious vector"

tern# ping -c5 172.16.1.2

PING 172.16.1.2 (172.16.1.2): 56 data bytes
64 bytes from 172.16.1.2: icmp_seq=0 ttl=128 time=0.911 ms
64 bytes from 172.16.1.2: icmp_seq=1 ttl=128 time=0.426 ms
64 bytes from 172.16.1.2: icmp_seq=2 ttl=128 time=0.436 ms
64 bytes from 172.16.1.2: icmp_seq=3 ttl=128 time=0.408 ms
64 bytes from 172.16.1.2: icmp_seq=4 ttl=128 time=0.451 ms

--- 172.16.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/std-dev = 0.408/0.526/0.911/0.193 ms

that interface is ok though...

tern# ping -c5 192.168.5.1

PING 192.168.5.1 (192.168.5.1): 56 data bytes
64 bytes from 192.168.5.1: icmp_seq=0 ttl=255 time=2268.892 ms
64 bytes from 192.168.5.1: icmp_seq=1 ttl=255 time=1263.132 ms
64 bytes from 192.168.5.1: icmp_seq=2 ttl=255 time=253.311 ms
64 bytes from 192.168.5.1: icmp_seq=3 ttl=255 time=1223.174 ms
64 bytes from 192.168.5.1: icmp_seq=4 ttl=255 time=213.342 ms

--- 192.168.5.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/std-dev = 213.342/1044.370/2268.892/761.007 ms

those times are very silly.

tern# traceroute 192.168.5.1
traceroute to 192.168.5.1 (192.168.5.1), 64 hops max, 40 byte packets  1 192.168.5.1 (192.168.5.1) 487.230 ms 784.17 ms 1972.439 ms

very, very silly.

tern# ifconfig hme1 down
tern# ping -c5 192.168.5.1
PING 192.168.5.1 (192.168.5.1): 56 data bytes

Jun  9 16:14:06 tern /bsd: hme2: device timeout
Jun  9 16:14:06 tern /bsd: hme2: device timeout
Can we help you?
In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software:
Learn more about Pantek's Technical Support offerings
Place an order for 24/7/365 Technical Support here
Learn more about Pantek, who we are, and what we do
Chat with a Live Operator
Related openbsd.org Links
advocacy
announce
bugs
ipv6
misc
ports
ports-changes
security-announce
smp
source-changes
tech
Request more information about Pantek
Jun  9 16:14:06 tern /bsd: hme2: device timeout

--- 192.168.5.1 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

and now it's just b0rked!

On the production system I not only had the very long round trip times, but when I had multiple networks on the QFE card, they'd all stop working with something like:

tern# ping 192.168.5.1
PING 192.168.5.1 (192.168.5.1): 56 data bytes ping: sendto: Host is down
ping: wrote 192.168.5.1 64 chars, ret=-1

I had this earlier on the test system too, but I can't reproduce it now.

This basically renders 3 ports on the quad card unusable - as long as only one port is used, it seems fine.

Any fix would be most appreciated :)

Thanks,

Doug.

>How-To-Repeat:

See above.

>Fix:

Unknown. Received on Tue Jun 10 05:57:09 2003