Wednesday 24 November 2010

Solaris 11: ipadm and dladm


Just as Solaris 10 was a quantum leap from Solaris 8 and 9, Solaris 11 too is a leap forward. You can now use ipadm and dladm to configure all the details you previous had to mess around with numerous files to get your configuration set.

Lets say we want two aggregations of two physical nic ports, with two addresses. Easy.

root@ramjet:~# dladm create-aggr -l bge0 -l bge1 aggr0
root@ramjet:~# dladm create-aggr -l bge2 -l bge3 aggr1

You can now see we have our aggregate devices

root@ramjet:~# dladm show-link


LINK        CLASS     MTU    STATE    BRIDGE     OVER
bge2        phys      1500   down     --         --
bge0        phys      1500   up       --         --
bge1        phys      1500   down     --         --
bge3        phys      1500   up       --         --
aggr0       aggr      1500   up       --         bge0 bge1
aggr1       aggr      1500   up       --         bge2 bge3

Lets add some addresses

root@ramjet:~# ipadm create-addr -T static -a 192.168.2.220/24 aggr0/v4static
root@ramjet:~# ipadm create-addr -T static -a 192.168.2.221/24 aggr1/v4static

You can see ipadm implicitly creates the 'if', based on the devices available and the 'addrobj' name

root@ramjet:~# ipadm show-if
IFNAME     STATE    CURRENT      PERSISTENT
lo0        ok       -m-v------46 ---
aggr0      ok       bm--------46 -46
aggr1      ok       bm--------46 -46

root@ramjet:~# ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
lo0/v4            static   ok           127.0.0.1/8
aggr0/v4static    static   ok           192.168.2.220/24
aggr1/v4static    static   ok           192.168.2.221/24
lo0/v6            static   ok           ::1/128

To finish, route -p add default 192.168.2.1 to set your default route.

4 comments:

Hanjo said...

Good wrap-up, thanks! However there seem to be some problems (at least for me). If I set it up as described everything works fine, but only until I restart the network/physical:default service (e.g. by restarting the system). After rebooting all the configuration looks exactly the same (ifconfig -a, dladm show-aggr, ipadm show-if, ipadm show-addr, netstat -rn), but no communication is possible anymore ("network unreachable"). The only thing that helps is to delete the interface and address using ipadm, delete the aggregate using dladm, rebooting and setting them up all over again.
Any idea how to solve this?

MV said...

I've double checked it on OpenIndiana and Sol11 Express and I've not seen the issue you mention.

It might be worth checking you've no legacy config (/etc/hostname., /etc/defaultrouter) and that you have a default route set (route -p add default ).

If you still see an issue, might be worth posting a bug!

Hanjo said...

Ok, I double-checked for legacy config and I'm pretty sure there was nothing left, but still it didn't work. So I reinstalled the system having all network wires unplugged and tried the above right after the installation finished - now it works.

Thanks for your help; I guess the service needs to become more robust ^^

Hanjo said...

I couldn't make it work; there was no legacy config. So I reinstalled the system - and voilĂ  - it worked.

Just yesterday I had a similar problem: the configuration seemed to be fine, but it just wouldn't communicate at all. It took me some time, but in the end I figured out that this is not a problem of the link aggregation, but of the stupid firewall, which has automatically been set to use the file /etc/nwam/loc/NoNet/ipf.conf by the damn (disabled) nwam service. This file basically has two rules:
block in log all
block out log all
Great, so in this case it has been the firewall that caused trouble. I cannot confirm, that this was also the reason for my problem before the reinstallation as I haven't considered this at that time (I didn't even enable the firewall). However it might be worth checking this. This command is handy for changing the path to the ipf.conf:
svccfg -s ipfilter:default setprop firewall_config_default/custom_policy_file = "/etc/ipf/ipf.conf"

Good Luck!