free blog   apache   shopping directory   php powered
stallion_workmbreitba: Infiniband is "stupid fast", IPoIB is like wrapping a PDU in bacon and shoving over IB
TommyTheKidNFS should use RDMA
lewellyni'm afraid to open the attachments :)
TommyTheKidI am not sure about iSCSI
balroglewellyn: what are the attachments?
mbreitbathey can both use RDMA, and will bypass the IP stack if I can get that configured properly, but I want to at least try to get the IPoIB working
TommyTheKidi want bacon
mbreitbaplus, the RDMA is even less well documented than the IPoIB
stallion_workmbreitba: be a man and use RDMA
lewellynbalrog: a .doc and a .pdf
lewellyni've apparently been invited to beijing in july
balrogI see
eviljamesbonsaikitten: I have found my next purchase.  Hats off to you! :D
balrogwhy are you afraid then? :)
mbreitbanot that IB is well documented at all
wrapsteris there a way i can figure out what internationalization support is provided by default in osol and what can be downloaded later?
rkeeneYou won't see the advantages of infiniband over 10GigE if you're using only IP over it.
bonsaikitteneviljames: really nice hardware, I like them :)
stallion_workmbreitba: its documented well enough.
lewellynbecause i don't have any ties to the sector, and it was sent with a return address over 163's mail servers
mbreitbathe other reason to use IB is that our bladecenter supports it, and the total cost was less than what 10GE would have costed to impliment
balrogopen it in google docs :)
TommyTheKidI would focus on RDMA, although we had talked about using IBoIP because its fanstastically cheaper than 10GIG (I think)
mbreitbawell, I want to get the simpler protocols working first, then move on to the RDMA
lewellyni'd still have to download the attachment ;)
lewellynhuh. bing says it's likely legit
mbreitbathat, and I don't know that the windows boxes that we're currently using work that well with NFSoRDMA
balrogI don't see how downloading the attachment could cause issues, unless you open it
mbreitbahere's the situation - we've got a large infrastructure running Windows/Hyper-V
mbreitbaEverything is on iSCSI right now, formatted NTFS
mbreitbawe want to move to either Xen or VMWare (haven't decided which yet)
TommyTheKidmbreitba: I would check the simple stuff first, dladm show-phys, make sure you have link, ifconfig -a .. make sure its UP, make sure it has a proper IP/netmask/broadcast, make sure there is no firewall (ipfilter?)
mbreitbawant to utilize storage on NFS volumes for data portability
TommyTheKidunfortunately not sure how any of that maps into nexenta :)
mbreitbaEverything that I've checked looks fine
mbreitbaall of the IB queries show that the interfaces are all connected and up
mbreitbaand the nexenta box is able to ping itself on the IB interface
TommyTheKidmake sure the d0ze side has no firewalls as well
mbreitbano firewalls on the Win side
TommyTheKidofcourse if they can ping eachother...
mbreitbaI'm about 20k deep into this project, and it's really starting to piss me off
stallion_workmbreitba: well... I wouldnt have high hopes for using just NFS
TommyTheKidso is your interface ibd0?
mbreitbaibd0 and ibd1 on the nexenta box
stallion_workyou are better off with a NAS head which supports CIFS/NFS access to the same filesystems
mbreitbawell, the nexenta box would be the NAS head end - I can provide CIFS storage out of it also
mbreitbaalso - more info about the build here : - details about the hardware and intentions
TommyTheKidok yuck, our IB router is running S10 :(
TommyTheKidIB router -> system with 10G and IB connecitivy, routing between them
mbreitbaI've already tried using bare OpenSolaris - I actually got the IBoIP working ok on it using 2009.06, but it would frequently just drop the connection
mbreitbaiSCSI over the IPoIB link would reset every 35 seconds (literally, like clockwork)
mbreitbaNFS wouldn't mount, but I'm wondering if that was because it was only offering it over RDMA
mbreitbaIPoIB wouldn't work at all in 134a
TommyTheKidyea, so I wish I had some great knowledge to impart, I know we are messing with IB in our lab, for the same reason you are, but I don't know where we are with it.. I was just asked for a subnet to route to :)
TommyTheKida subnet and a route to it on serverX that is
TommyTheKidand the reason I grumbled about S10 was nmap isnt there ;)
mbreitbawell, maybe it's time to jump back into OpenSolaris and hack on it from that end.....the Nexenta stuff is a little more black-box'ish and that seems to be complicating things
lewellynTommyTheKid: er... yes it is
lewellyninstall the companion cd :P
rkeeneAtleast you aren't trying to do IP-over-Fibre-Channel in Solaris 10.  It's completely broken and Sun/Oracle refuses to fix it.
TommyTheKidI am not going to go installing software on their router
TommyTheKidIBoFC, yikes
lewellynyour statement is still false ;)
rkeeneIBoFC ?
rkeenefcip(7D) is completely broken
mbreitbaNexenta actually told me that the IB card that we had wasn't on the OpenSolaris HCL......which was totally wrong, in any event though, I'm hoping to get something running
TommyTheKidlewellyn: its not there by default, companion CD (/opt/sfw I assume?) is generally hopelessly out of date, so we stopped installing it
mbreitbahow is the CIFS support on OpenSolaris - is it going to work for my Windows hosts until I can get migrated to Xen/VMWare and use NFS?
TommyTheKidsmrt: explain cifs
smrt File sharing protocol commonly used on Windows.   OpenSolaris includes two cifs servers, one builtin to the kernel, and the samba userspace daemon.   (See also: cifs server, samba)
lewellynTommyTheKid: it may be out of date, but it is part of a "complete" install. hell, most of the os is hopelessly out of date ;)
TommyTheKidthats true lewellyn  :)
TommyTheKidI used to install it once, centrally and mount it NFS.. but I don't know how long its been since I had that :)
mbreitbaand I'm guessing that if I use CIFS, I can export that via NFS later and Xen/VMWare will still be able to read it?
TommyTheKidsmrt: explain cifs server
smrt OpenSolaris has a new CIFS (Windows file sharing in SMB) server in the kernel. Good resources for it include and . The CIFS provides better performance and easier administration for CIFS than Samba but Samba provides other SMB services (See also: samba)
lewellynmbreitba: if you use the kernel cifs, you can manage nfs and cifs with the same interface
mbreitbaalrighty, I think my answer is to turn back to OpenSolaris then and hack some more on it
mbreitbaalso, just out of curiosity, as I didn't find this in my digging - if the NFS server detects RDMA capable hardware, does it only offer NFS over RDMA, or does it offer it up over IP at the same time?
TommyTheKidI *think* if both ends are on IB, it skips over the IP stack (uses RDMA)
TommyTheKidbut maybe thats because it doesn't offer NFS over IP on IB interfaces,,, I couldn't tell you there ;)
mbreitbaalrighty - thanks for the info - back to hacking on OpenSolaris - maybe I just need a bigger axe ;)
TommyTheKidmbreitba: good luck :)
TommyTheKidi dont honestly know why they used S10, probably to keep from scaring the OPS folks :p
TommyTheKid... apparently either the routing is messed up or they are firewalling, either way, I am not mucking with their config :)
lewellynhrm. has anyone packaged tortoisehg for opensolaris yet? it seems overly logical to have ;)
TommyTheKidlewellyn: quick question... if I build a package with SFE, can I like scp it to another system and install it?
TommyTheKidnot that I hafe SFE working yet ;)
alanclewellyn: GUI's are illogical though
Action: LadySerena is a happy kitty
TonnerreLadySerena, good!
TommyTheKidi suppose thats better than beeing a sad salamander ?
LadySerenaAt school, I got my pin and senior scrubs today.  :D
lewellynTommyTheKid: if you build an svr4 package.
lewellynalanc: they're still faster than opengrok at getting history ;)
TommyTheKidlewellyn: otherwise, I can just point at the pkg server that will somehow supposedly be running there on that machine and install it over the network?
lewellynand if you're poking around in nautilus, it's fater than opening a terminal and navigating
lewellynTommyTheKid: in theory. i've not built ips yet ;)
TommyTheKidthis could be interseting :)
TommyTheKidinteresting even
lewellynremember last night how i was asking about pkgtoolrc? ;)
TommyTheKidhmm, I was bashing my head on the bootstrap script, so I didnt see it :)
lewellyni still don't know why you were editing it.
lewellynjust run the damn thing :P
TommyTheKidit doesnt work
lewellynsmrt: explain doesn't work
smrt Then make it file for unemployment benefits.
TommyTheKidI have an NFS home directory and it doesn't cooperate
TommyTheKidso I tried to edit the USERHOMEDIR= setting to point at /ws/tm.... and that sorta works, except that it still somehow makes some stuff appear in ~/
lewellynyeah. i keep meaning to patch it to deal with nfs homedirs.
lewellynbut tomww is quiet and i don't know how out-of-date the current download is vs his local copy
TommyTheKidif he doesn't commit his copy up, then its not there :P
lewellynpatches are less useful when they're against virtually every line and you're trying to commit them against something fairly different ;)
--- Thu Apr 29 2010
Popular searches: TOR directory authorities   machine local guest domain   anything-config   

Generated by 2.1mg by Jeff Waugh - return