[Open-FCoE] [PATCH] open-fcoe: Initial code for fcoemon and Open-FCoE service script

Steve Ma steve.ma at intel.com
Thu Jan 22 20:39:33 UTC 2009


fcoemon is the daemon of the fcoe system service to interact with
the dcbd daemon (Data Center Bridging daemon) and the kernel
RT_NETLINK service. When fcoemon starts, it establishes a socket
connection with the dcbd daemon. It then sends commands to, and
receives responses and event from the DCB daemon.  The RT_NETLINK
service reports the link status of the Ethernet ports to the fcoemon.
The fcoemon will process the responses and events received from
dcbd and invoke the fcoeplumb script to add multiq queue descipline,
to add/remove/replace the skbedit filters, and create/destroy the
FCoE interfaces.

The "Open-FCoE service script" is the bash script, /etc/init.d.fcoe,
after the Open-FCoE user-space package is installed. This script,
just like the other system service scripts, starts, stops, restarts,
and shows status of the fcoe service. The script will create FCoE
interface for an Ethernet port directly using fcoeadm tool if the
port is not configured to use DCB service. Otherwise, if the port
is configured to use DCB service, it will start the fcoemon daemon,
and let the fcoemon to add multiq qdiscipline and skbedit filter.
The man page of fcoemon describes more in this area.

The configuration files include
    /etc/fcoe/config
    /etc/fcoe/cfg-eth0, ..., /etc/fcoe/cfg-eth<n> one for each
    Ethernet port.
These files are self-documented, and must be edited to contain the
settings for your environment.

Notice that the fcoe system service does not depend on dcbd service.
However, the fcoemon daemon will be started by the fcoe service only
if any one of the Ethernet port requires DCB service. In this case,
fcoe service depends on dcbd service. If none of the Ethernet ports
requires DCB service, the fcoemon will not be started and, in this
case, the fcoe service does not depend on dcbd service.

With this patch applied on the Open-FCoE user-space source tree, the
installation should contain the following files:

    /etc/init.d/fcoe                  startup script
    /etc/fcoe/config                  Common configuration file
                                       (currently only used to specify
                                       debugg options)
    /etc/fcoe/cfg-eth*                Per-port configuration files
    /etc/fcoe/scripts/fcoeplumb       Script only invoked by fcoemon
                                       to add qdisc, filters and
                                       create/destroy FCoE interfaces.
    /sbin/fcoeadm                     FCoE management tool
    /sbin/fcoemon                     FCoE monitor
    /usr/share/man/man8/fcoeadm.8.gz  Man-page for fcoeadm
    /usr/share/man/man8/fcoemon.8.gz  Man-page for fcoemon

For installation, please see the INSTALL file included in the patch.

The Open-FCoE service script has not been well-developed, and heavily
tested. More patches are expected to be available in the near future.
The DCB package is available to be downloaded from the sourceForge web site.

Signed-off-by: Steve Ma <steve.ma at intel.com>
---

 usr/INSTALL                         |   59 +
 usr/Makefile                        |   29 
 usr/doc/fcoeadm.8                   |  100 +-
 usr/doc/fcoemon.8                   |  215 ++++
 usr/etc/fcoe.conf                   |    3 
 usr/etc/initd/cfg-ethx              |    9 
 usr/etc/initd/config                |   10 
 usr/etc/initd/initd.fcoe            |   12 
 usr/etc/initd/initd.suse            |  376 ++++++
 usr/tools/fcoeadm/Makefile          |    2 
 usr/tools/fcoeadm/fcoeadm.c         |   25 
 usr/tools/fcoeadm/fcoeadm.h         |    8 
 usr/tools/fcoeadm/fcoeadm_display.c |    2 
 usr/tools/fcoemon/Makefile          |  155 +++
 usr/tools/fcoemon/fcoemon.c         | 2042 +++++++++++++++++++++++++++++++++++
 usr/tools/fcoemon/fcoemon.h         |  115 ++
 usr/tools/fcoemon/fcoemon_utils.c   |  760 +++++++++++++
 usr/tools/fcoemon/fcoemon_utils.h   |  315 +++++
 usr/tools/fcoemon/fcoeplumb.sh      |  345 ++++++
 19 files changed, 4461 insertions(+), 121 deletions(-)
 create mode 100644 usr/doc/fcoemon.8
 delete mode 100644 usr/etc/fcoe.conf
 create mode 100644 usr/etc/initd/cfg-ethx
 create mode 100644 usr/etc/initd/config
 mode change 100644 => 100755 usr/etc/initd/initd.fcoe
 create mode 100755 usr/etc/initd/initd.suse
 create mode 100644 usr/tools/fcoemon/Makefile
 create mode 100644 usr/tools/fcoemon/fcoemon.c
 create mode 100644 usr/tools/fcoemon/fcoemon.h
 create mode 100644 usr/tools/fcoemon/fcoemon_utils.c
 create mode 100644 usr/tools/fcoemon/fcoemon_utils.h
 create mode 100755 usr/tools/fcoemon/fcoeplumb.sh

diff --git a/usr/INSTALL b/usr/INSTALL
index 23f5e3a..f57b400 100644
--- a/usr/INSTALL
+++ b/usr/INSTALL
@@ -3,12 +3,16 @@
 
 The FCoE Management Tools included in this package are
 
-        fcoeadm - program to create, destroy, and to display FCoE interfaces
+        fcoeadm - program to create, reset, destroy, and display FCoE interfaces
+        fcoemon - program to monitor the events from the DCB daemon
 
-The HBAAPI library and the HBAAPI vendor library source must be build and
-installed before you can build the management tools. The HBAAPI vendor
+Requirements:
+
+        The HBAAPI library and the HBAAPI vendor library source must be build
+and installed before you can build the management tools. The HBAAPI vendor
 library, libhbalinux, may be downloaded from www.Open-FCoE.org. The instructions
-in the package describes how to download and build the libraries.
+in the package describes how to download and build the libraries.  See the man
+pages for other requirements.
 
 Build and Install The Management Tools
 --------------------------------------
@@ -19,39 +23,40 @@ Build and Install The Management Tools
 
    To build the tools,
 
-       make LIBHBALINUX=<path-to-libhbalinux-source>
-       make LIBHBALINUX=<path-to-libhbalinux-source> install
+       make LIBHBALINUX=<path-to-libhbalinux-source> \
+            DCB_SRC=<path-to-DCB-source>
+       make LIBHBALINUX=<path-to-libhbalinux-source> \
+            DCB_SRC=<path-to-DCB-source> install
+
+    For example,
+
+        make LIBHBALINUX=/var/work/libhbalinux \
+             DCB_SRC=/var/work/dcb/dcbd-0.7.31rt
 
    To cleanup, issue
 
        make clean
        make uninstall
 
+   Notice that before issuing "make uninstall", make sure the fcoe service is not
+   running.
+
 2) The installation adds the following files to the system,
 
        /sbin/fcoeadm
+       /sbin/fcoemon
        /usr/share/man/man8/fcoeadm.8.gz
-       /etc/fcoe/open-fcoe.conf
-       /etc/rc.d/init.d/open-fcoe
-
-3) Running the tools require to have the libHBAAPI.so and libhbalinux.so,
-   and /etc/hba.conf installed on the system. See the INSTALL instructions
-   of HBAAPI Vendor Libraries.
-
-4) If you have edited the file /etc/fcoe/open-fcoe.conf to include
-   one and only one line of
-
-       HBA=<ethX>
-
-   where <ethX> is the ethernet interface you will use to create FCoE
-   instance, you may start and stop the open-fcoe service with the
-   following commands:
-
-       # service open-fcoe start
-       # service open-fcoe stop
+       /usr/share/man/man8/fcoemon.8.gz
+       /etc/fcoe/config
+       /etc/fcoe/cfg-ethX (one file per ethX port)
+       /etc/fcoe/scripts/fcoeplumb
+       /etc/init.d/fcoe
 
-       # service open-fcoe
-       Usage: /etc/init.d/open-fcoe {start|stop|reload|restart|status}
 
-   The fcoeadm tool requires the open-fcoe service to be started.
+	The /etc/fcoe is default configuration files location but this can be changed
+	by updating this in usr/doc/fcoemon.8 and CONFIG_DIR variable in
 
+	etc/initd/initd.suse
+	tools/fcoemon/fcoemon.c
+	tools/fcoemon/fcoeplumb.sh
+	tools/fcoemon/Makefile
diff --git a/usr/Makefile b/usr/Makefile
index 3412bdd..d4b5332 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -16,16 +16,7 @@ ifneq "$(filter-out $(LEGAL_OS), $(shell uname -s))" ""
     $(error bad build OS $(shell uname -s))
 else
 
-# set the following corresepondingly to your preferred locations
-DESTDIR ?=
-
-ETCDIR = /etc
-INITDDIR = $(ETCDIR)/init.d
-
-INSTALL = install
 MAKE = make
-GZIP = gzip
-RM = rm
 
 default: all
 
@@ -35,27 +26,11 @@ all:
 clean:
 	@$(foreach i, $(wildcard tools/*), $(MAKE) -C $(i) clean ; )
 
-install: install_initd
+install:
 	@$(foreach i, $(wildcard tools/*), $(MAKE) -C $(i) install ; )
 
-uninstall: uninstall_initd
+uninstall:
 	@$(foreach i, $(wildcard tools/*), $(MAKE) -C $(i) uninstall ; )
 
-install_initd:
-	@$(INSTALL) -d $(DESTDIR)$(ETCDIR)/fcoe
-	@$(INSTALL) -v -m 644 etc/fcoe.conf $(DESTDIR)$(ETCDIR)/fcoe/open-fcoe.conf
-	@$(INSTALL) -d $(DESTDIR)$(INITDDIR)
-	@$(INSTALL) -v -m 755 etc/initd/initd.fcoe $(DESTDIR)$(INITDDIR)/open-fcoe
-	@echo
-	@echo -------------------------------------------------
-	@echo Please check INSTALL file for detailed information.
-	@echo Remember to setup the FCoE config file located at:
-	@echo -e "\t$(DESTDIR)$(ETCDIR)/fcoe/open-fcoe.conf"
-	@echo
-
-uninstall_initd:
-	@$(RM) -vf $(DESTDIR)$(ETCDIR)/fcoe/open-fcoe.conf
-	@$(RM) -vf $(DESTDIR)$(INITDDIR)/open-fcoe
-
 endif
 endif
diff --git a/usr/doc/fcoeadm.8 b/usr/doc/fcoeadm.8
index 4d91b39..09731f2 100644
--- a/usr/doc/fcoeadm.8
+++ b/usr/doc/fcoeadm.8
@@ -1,4 +1,4 @@
-.TH FCOEADM 8 "August 21, 2008" "Open-FCoE Applications" "Open-FCoE Tools"
+.TH "FCOEADM" "8" "November 4, 2008" "Open-FCoE Applications" "Open-FCoE Tools"
 .SH "NAME"
 \fBfcoeadm\fR \- The Fibre Channel over Ethernet (FCoE) administration tool
 .SH "SYNOPSIS"
@@ -16,6 +16,8 @@
 .P
 \fBfcoeadm\fR [\fB\-s\fR|\fB\-\-stats\fR] [\fI<ethX>\fR [\fB\-n\fR \fI<interval>\fR]]
 .P
+\fBfcoeadm\fR [\fB\-v\fR|\fB\-\-version\fR]
+.P
 \fBfcoeadm\fR [\fB\-h\fR|\fB\-\-help\fR]
 .SH "DESCRIPTION"
 The \fBfcoeadm\fR command is intended to be the FCoE management tool for the Linux systems.
@@ -23,24 +25,24 @@ The \fB\-c\fR, \fB\-d\fR, and \fB\-r\fR options are used to create, destroy, and
 an FCoE instance on a given network interface.  The other options are used to query the
 information of the FCoE instance which includes the interface information, target information,
 LUN information, and port statistics.  The \fBfcoeadm\fR command invokes the \fBHBAAPI\fR library
-routines to obtain these information.  The \fBHBAAPI\fR library routines invoke the vendor-specific
+routines to obtain these information.  The \fBHBAAPI\fR library routines invoke the vendor\-specific
 library, \fBlibhbalinux\fR, to grab the information from the /sys file system.  In other words,
 the \fBfcoeadm\fR command requires to have \fBlibHBAAPI\fR and \fBlibhbalinux\fR installed on the system to work.
-The \fBlibhbalinux\fR is maintained at \fB\fIwww.Open-FCoE.org\fR. The installation instructions of
+The \fBlibhbalinux\fR is maintained at \fB\fIwww.Open\-FCoE.org\fR. The installation instructions of
 \fBlibhbalinux\fR also instructs how to download the \fBHBAAPI\fR source code, build and install with
 the \fBlibhbalinux\fR.  The last option \fB\-h\fR is used to show a brief usage message of the supported
 command syntax.
 .SH "OPTIONS"
 .TP
 \fB\-c\fR, \fB\-\-create\fR \fI<ethX>\fR
-Creates an FCoE instance based on the given \fI<ethX>\fR. It first checks the 
-existence of /sys/module/fcoe/ to verify that the fcoe module is loaded. Then 
+Creates an FCoE instance based on the given \fI<ethX>\fR. It first checks the
+existence of /sys/module/fcoe/ to verify that the fcoe module is loaded. Then
 the \fI<ethX>\fR is written to /sys/module/fcoe/create.
 .TP
 \fB\-d\fR, \fB\-\-destroy\fR \fI<ethX>\fR
 Destroys an FCoE instance based on the given \fI<ethX>\fR. It first checks the
 existence of /sys/module/fcoe/ to verify that fcoe module is loaded. Then
-the \fI<ethX>\fR is written to /sys/module/fcoe/destroy. 
+the \fI<ethX>\fR is written to /sys/module/fcoe/destroy.
 .TP
 \fB\-r\fR, \fB\-\-reset\fR \fI<ethX>\fR
 Resets the fc_host associated with the FCoE interface given by \fI<ethX>\fR.
@@ -57,7 +59,7 @@ show the information of all the discovered targets from all the FCoE instances c
 .TP
 \fB\-l\fR, \fB\-\-lun\fR \fI<target_port_id>\fR \fI<lun_id>\fR
 Show the detailed information of a specific LUN with \fI<lun_id>\fR at the target
-with port id \fI<target_port_id>\fR. port id is also known as FC-ID. If \fI<lun_id>\fR
+with port id \fI<target_port_id>\fR. port id is also known as FC\-ID. If \fI<lun_id>\fR
 is not specified, all the LUNs associated with the target will be shown.
 .TP
 \fB\-s\fR, \fB\-\-stats\fR \fI<ethX>\fR \fB\-n\fR \fI<interval>\fR
@@ -66,81 +68,83 @@ The information will be display in one line on the screen per given time interva
 be specified in whole intergers greater than 0. It specifies the time interval in the unit of second.
 If \fI<interval>\fR is not specified, the default interval is one second.
 .TP
-\fB\-h\fR, \fB\-\-help\fR \fI<ethX>\fR
-Displays this help information.
-.TP 
-\fI<ethX>\fR
-The network interface name, such as eth0, eth1, etc.
+\fB\-v\fR, \fB\-\-version\fR
+Displays the version of the \fBfcoeadm\fR command.
+.TP
+\fB\-h\fR, \fB\-\-help\fR
+Displays the usage message of the \fBfcoeadm\fR command.
+.TP
+where \fI<ethX>\fR is the network interface name, such as eth0, eth1, etc.
 .SH "EXAMPLES"
 Creates an FCoE instance on eth2
-.IP 
-$ \fBfcoeadm\fR -c eth2
+.IP
+$ \fBfcoeadm\fR \-c eth2
 .P
 Destroys the FCoE instance on eth2
-.IP 
-$ \fBfcoeadm\fR -d eth2
+.IP
+$ \fBfcoeadm\fR \-d eth2
 .P
 Resets the FCoE instance on eth2
-.IP 
-$ \fBfcoeadm\fR -r eth2
+.IP
+$ \fBfcoeadm\fR \-r eth2
 .P
 Show the information of all the adapters and their ports having FCoE instances created.
-.IP 
-$ \fBfcoeadm\fR -i
+.IP
+$ \fBfcoeadm\fR \-i
 .P
 Show the information of a specific interface eth3.  If eth3 has no FCoE instances created,
 the command will show the error "No fc_host found for eth3".
-.IP 
-$ \fBfcoeadm\fR -i eth3
+.IP
+$ \fBfcoeadm\fR \-i eth3
 .P
 Show the information of all the discovered targets from all the ports having FCoE instances
 created (they may be on different adapter cards). A brief listing of discovered LUNs are
-listed after the target they are associated with, if any. 
-.IP 
-$ \fBfcoeadm\fR -t
+listed after the target they are associated with, if any.
+.IP
+$ \fBfcoeadm\fR \-t
 .P
 Show the information of all the discovered targets from a given port (eth3) having FCoE instance
 created. A brief listing of discovered LUNs are listed after each target they are associated with, if any.
-.IP 
-$ \fBfcoeadm\fR -t eth3
+.IP
+$ \fBfcoeadm\fR \-t eth3
 .P
 Show the detailed information of all the LUNs associated with a specific target.
-The target is identified by its port id (aka FC-ID) 0xD700EF.
-.IP 
-$ \fBfcoeadm\fR -l 0xD700EF 
+The target is identified by its port id (aka FC\-ID) 0xD700EF.
+.IP
+$ \fBfcoeadm\fR \-l 0xD700EF
 .P
-.IP 
-$ \fBfcoeadm\fR -l D700EF 
+.IP
+$ \fBfcoeadm\fR \-l D700EF
 .P
-.IP 
-$ \fBfcoeadm\fR -l 0xd700ef 
+.IP
+$ \fBfcoeadm\fR \-l 0xd700ef
 .P
-.IP 
-$ \fBfcoeadm\fR -l d700ef 
+.IP
+$ \fBfcoeadm\fR \-l d700ef
 .P
 Show the detailed information of a LUN associated with a specific target.
-The target is identified by its port id (aka FC-ID) 0xD700EF and the LUN
+The target is identified by its port id (aka FC\-ID) 0xD700EF and the LUN
 is identified by its LUN id.
-.IP 
-$ \fBfcoeadm\fR -l 0xD700EF 1
+.IP
+$ \fBfcoeadm\fR \-l 0xD700EF 1
 .P
 Show the statistics information of a specific port eth3 having FCoE instances created.
 The statistics are displayed one line per time interval. The default interval is one
-second if -n option is not specified.
-.IP 
-$ \fBfcoeadm\fR -s eth3
+second if \-n option is not specified.
+.IP
+$ \fBfcoeadm\fR \-s eth3
 .P
-.IP 
-$ \fBfcoeadm\fR -s eth3 -n 3
+.IP
+$ \fBfcoeadm\fR \-s eth3 \-n 3
 .P
-.IP 
-$ \fBfcoeadm\fR -s eth3 -n3
+.IP
+$ \fBfcoeadm\fR \-s eth3 \-n3
 .SH "REPORTING BUGS"
 If you have identified a
 defect please either file a bug or engage the development mailing list at
-<http://www.Open-FCoE.org>.
+<http://www.Open\-FCoE.org>.
 .SH "SUPPORT"
-Open-FCoE is maintained at <http://www.Open-FCoE.org>. There are resources
+Open\-FCoE is maintained at <http://www.Open\-FCoE.org>. There are resources
 available for both developers and users at that site.
 
 
diff --git a/usr/doc/fcoemon.8 b/usr/doc/fcoemon.8
new file mode 100644
index 0000000..2b0187a
--- /dev/null
+++ b/usr/doc/fcoemon.8
@@ -0,0 +1,215 @@
+.TH "FCOEMON" "8" "December 22, 2008" "Open-FCoE Applications" "Open-FCoE Tools"
+.SH "NAME"
+\fBfcoemon\fR \- The Fibre Channel over Ethernet (FCoE) administration tool
+          for monitoring and processing events from DCB daemon.
+.SH "SYNOPSIS"
+\fBfcoemon\fR [\fB\-h\fR | \fB\-\-help\fR]
+.P
+\fBfcoemon\fR [\fB\-v\fR | \fB\-\-version\fR]
+.P
+\fBfcoemon\fR [\fB\-d\fR | \fB\-\-debug\fR] [\fB\-e\fR | \fB\-\-exec\fR \fI<path\-to\-fcoeplumb\-script>\fR]
+.SH "DESCRIPTION"
+The \fBfcoemon\fR command is a FCoE management tool provided by the Open\-FCoE package.
+\fBfcoemon\fR is the daemon of the \fBfcoe\fR system service.  When \fBfcoemon\fR starts, it establishes a socket
+connection with the \fBDCB\fR daemon. It then sends commands to, and receives responses
+and event from the \fBDCB\fR daemon. The \fBfcoemon\fR  will process the received responses
+and events and invoke the \fBfcoeplumb\fR script to create and destroy the FCoE interfaces.
+Since the \fBfcoemon\fR depend on the existence of \fBDCB service\fR, there
+are settings required for \fBDCB\fR before \fBfcoemon\fR can be started. See
+the \fIDCB Settings\fR section below.
+
+Notice that the fcoe system service does not depend on dcbd service. However, the fcoemon daemon will be
+started by the fcoe service only if any one of the Ethernet port requires DCB service. In this case, fcoe
+service depends on dcbd service.  If none of the Ethernet ports requires DCB service, the fcoemon will not
+be started and, in this case, the fcoe service does not depend on dcbd service.
+
+.SH "OPTIONS"
+.TP
+\fB\-h | \-\-version\fR
+Show the usage message of the \fBfcoeemon\fR command.
+.TP
+\fB\-v | \-\-version\fR
+Show the version of the \fBfcoeemon\fR command.
+.TP
+\fB\-d | \-\-debug\fR
+Enable debugging messages.
+.TP
+\fB\-e | \-\-exec\fR \fI<path\-of\-fcoeplumb\-script>\fR
+Specify the location of the \fBfcoeplumb\fR script.
+.SH "TERMINOLOGY"
+.TP
+\fBPFC\fR \- The DCB Priority Flow Control feature.
+.TP
+\fBApp:FCoE\fR \- The DCB Fibre Channel over Ethernet feature.
+.TP
+\fBLLINK\fR \- The DCB Logical Link TLV (or Logical Link) feature.
+.TP
+\fBmultiq\fR \- See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.
+.TP
+\fBskbedit\fR \- See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.
+.SH "INSTALLATION REQUIREMENTS"
+The linux kernel must be at version 2.6.27.1\-2 beta6 or higher. The DCB and FCoE kernel
+configuration options must be enabled. Both the linux kernel and iproute2 must support multiq
+and skbedit.  The DCB must be installed with version 0.9.2 and higher.
+.SH "SUPPORTED DCB EVENTS"
+In response to each supported event from the DCB daemon, the fcoemon collects the current
+settings from the DCB daemon and decide whether to delete and re-add the multiq queue discipline
+and skbedit filter. The \fBfcoemon\fR does not destroy, reset, or create FCoE interfaces during
+the DCB event processing.
+.TP
+\fBFEATURE_APP\fR
+If an event message is received from dcbd and if the feature code in the event message
+is FEATURE_APP (5), and if the subtype field is APP_FCOE_STYPE (0), we got a mode or
+configuration change event of the FCoE application.  The fcoemon will then issue queries
+to the DCB daemon to collect the current mode and configuration information.
+.TP
+\fBFEATURE_PFC\fR
+If an event message is received from dcbd and if the feature code in the event message
+is FEATURE_PFC (3), we got a mode or configuration change event of the Priority Flow
+Control (PFC) feature.  The fcoemon will then issue queries to the DCB daemon to collect
+the current mode and configuration information.
+.TP
+\fBFEATURE_LLINK\fR
+If an event message is received from dcbd and if the feature code in the event message
+is FEATURE_LLINK (6), and if the subtype field is LLINK_FCOE_STYPE (0), we got a mode
+or configuration change event of the Logical Link TLV feature.  The fcoemon will then
+issue queries to the DCB daemon to collect the current mode and configuration information.
+.SH "CRITERIA OF CREATING, RESETTING  AND DESTROYING FCOE INTERFACE"
+In this section the \fBdcbtool\fR is used to describe the conditions of the DCB feture status
+beccause the meaning is more understandable and precise. Although you may also issue the
+commands at run-time, the commands are intended only to be used for description purpose.
+.TP
+\fBPFC and App:FCoE\fR
+DCB is configured correctly if
+.RS
+.PD 0
+.TP 3
+.BR "1)" "  The command \fBdcbtool gc ethX dcb\fR shows DCB State: on"
+.TP 3
+.BR "2)" "  The command \fBdcbtool gc ethX app:0\fR shows Enable:true,"
+.TP 3
+.BR "  " "      Advertise:true, Willing:true."
+.TP 3
+.BR "3)" "  The command \fBdcbtool go ethX app:0\fR shows OperMode:true."
+.TP 3
+.BR "4)" "  The command \fBdcbtool go ethX pfc\fR shows OperMode:true and"
+.TP 3
+.BR "  " "      the values of pfcup."
+.TP 3
+.BR "5)" "  The command \fBdcbtool go ethX app:0\fR shows appcfg. The bits"
+.TP 3
+.BR "  " "      set to 1 are also set to 1 in pfcup found in (4)."
+.PD
+.RE
+.TP
+\fBLogical Link TLV\fR
+The Logical Link TLV feature is configured correctly if
+.RS
+.PD 0
+.TP 3
+.BR "1)" "  The command \fBdcbtool gc ethX ll:0\fR shows"
+.TP 3
+.BR "  " "      Enable:true, Advertise:true, Willing:true."
+.TP 3
+.BR "2)" "  The command \fBdcbtool go ethX ll:0\fR shows OperMode:true."
+.TP 3
+.BR "3)" "  The command \fBdbtool gp ethX ll:0\fR shows Link Status:up."
+.PD
+.RE
+.TP
+\fBCriteria to create FCoE interface\fR
+If DCB is required at the Ethernet port, a FCoE interface may be created only if
+the DCB and the Logical Link TLV feature are configured correctly. If DCB is not
+required at the Ethernet port, the FCoE interface may be created. FCoE interfaces
+are normally created by the \fBfcoe\fR system service.
+.TP
+\fBCriteria to Destroy FCoE Interface\fR
+An FCoE interface will only be destroyed when the \fBfcoe\fR system service is stopped.
+.TP
+\fBCriteria to reset a FCoE interface\fR
+The \fBfcoe\fR system service does not reset any FCoE interfaces.
+.TP
+\fBChanging DCB Configuration, Qdisc and Filters\fR
+Changing the DCB configuration, qdisc, and filter are considered to be administrative actions.
+When the \fBfcoe\fR system service starts up, it sets up the default DCB configuration, qdisc, and filter
+for reliable FCoE operations. Administrators may alter the configuration while the service is running.
+Changing the DCB parameters may cause the fcoemon daemon to delete the existing multiq queue discipline,
+skbedit filter and re-add; but the fcoe service will not touch (e.g. destroy or reset) the FCoE interface.
+Changing the DCB configuration, qdisc, and filter should be avoided while I/O traffic are in progress.
+.SH "FILES"
+The Installation of the Open-FCoE management tools include the following files:
+.TP
+\fB/etc/fcoe/config\fR
+This is the common configuration file for the \fBfcoe\fR system service. This file will
+be read by the \fI/etc/init.d/fcoe\fR" script, the \fI/etc/fcoe/scripts/fcoeplumb\fR script,
+and the \fIfcoemon\fR daemon. The default options in this file are:
+\fBDEBUG="yes"\fR and \fBUSE_SYSLOG="yes"\fR. The former is used to enable (select yes) or disable (select no)
+debugging messages of fcoemon, the \fIfcoe\fR service script, and the fcoeplumb script. The latter is
+to indicate if the log messages of fcoemon, the \fIfcoe\fR service script, and the fcoeplumb script are
+to be output to the system log.  Use editor to set the desired \fByes/no\fR values.
+.TP
+\fB/etc/fcoe/cfg-ethX\fR
+There is one of this file for each Ethernet interface \fBethX\fR found in the output of
+\fBcat /proc/net/dev\fR at the time of installation. This file will be read by the
+\fI/etc/init.d/fcoe\fR script and the \fIfcoemon\fR daemon.  The default options in this file are:
+\fBFCOE_ENABLE="no"\fR and \fBDCB_REQUIRED="yes"\fR.  The former is used to enable (select yes) or
+disable (select no) the FCoE service at the ethX port. The latter is to indicate if the DCB service
+is required (select yes) or not required (select no) at the ethX port. If the former is set to no, the
+latter is ignored. The selection of the settings should match the settings of the FCoE switch port connected
+to the local Ethernet ethX port. Use editor to set the desired \fByes/no\fR
+values for the \fBethX\fR interfaces.
+.TP
+\fB/etc/init.d/fcoe\fR
+This is the \fIfcoe\fR system service shell script. This script is invoked by the \fBinit\fR process
+or by the \fBservice\fR command.
+.TP
+\fB/sbin/fcoemon\fR
+This is the \fIfcoemon\fR daemon only invoked by the \fIfcoe\fR system service script.
+.TP
+\fB/sbin/fcoeadm\fR
+This is the program used by the \fIfcoe\fR system service to create or destroy FCoE interfaces.
+.TP
+\fB/etc/fcoe/scripts/fcoeplumb\fR
+This is a script only used by the \fBfcoemon\fR daemon.
+.TP
+\fB/usr/share/man/man8/fcoemon.8.gz\fR
+This is the man page of the \fBfcoemon\fR daemon (This file).
+.TP
+\fB/usr/share/man/man8/fcoeadm.8.gz\fR
+This is the man page of the \fBfcoeadm\fR program.
+.SH "FCOE SYSTEM SERVICE"
+If the \fIfcoe\fR system service is enabled, it will be started after the
+dcb service is started when the Linux system boots up. The following are the
+sample commands at system run-time for the \fIfcoe\fR system service.
+.TP
+\fBchkconfig --list fcoe\fR
+To check if the service is enabled.
+.TP
+\fBchkconfig fcoe off\fR
+To disable the \fBfcoe\fR system service.
+.TP
+\fBchkconfig fcoe on\fR
+To enable the \fBfcoe\fR system service.
+.TP
+\fBservice fcoe stop\fR
+To stop the \fBfcoe\fR system service.
+.TP
+\fBservice fcoe start\fR
+To start the stopped \fBfcoe\fR system service.
+.TP
+\fBservice fcoe restart\fR
+To stop and then start the \fBfcoe\fR system service.
+.TP
+\fBservice fcoe status\fR
+To check the status of the \fBfcoe\fR system service.
+.TP
+\fBservice fcoe\fR
+To show the command-line options of the \fBfcoe\fR system service.
+.SH "REPORTING BUGS"
+If you have identified a defect please either file a bug or engage the
+development mailing list at <http://www.Open\-FCoE.org>.
+.SH "SUPPORT"
+Open\-FCoE is maintained at <http://www.Open\-FCoE.org>. There are resources
+available for both developers and users at that site.
+
+
diff --git a/usr/etc/fcoe.conf b/usr/etc/fcoe.conf
deleted file mode 100644
index b6a2b39..0000000
--- a/usr/etc/fcoe.conf
+++ /dev/null
@@ -1,3 +0,0 @@
-# Put the interface name that you want to use for FCoE as below, for example,
-# HBA=eth2
-HBA=eth0
diff --git a/usr/etc/initd/cfg-ethx b/usr/etc/initd/cfg-ethx
new file mode 100644
index 0000000..1899a98
--- /dev/null
+++ b/usr/etc/initd/cfg-ethx
@@ -0,0 +1,9 @@
+## Type:       yesno
+## Default:    no
+# Enable/Disable FCoE service at the Ethernet port
+FCOE_ENABLE="no"
+
+## Type:       yesno
+## Default:    yes
+# Indicate if DCB service is required at the Ethernet port
+DCB_REQUIRED="yes"
diff --git a/usr/etc/initd/config b/usr/etc/initd/config
new file mode 100644
index 0000000..3c4360a
--- /dev/null
+++ b/usr/etc/initd/config
@@ -0,0 +1,10 @@
+## Type:       yesno
+## Default:    yes
+# Switch on/off debug messages (script & C code)
+DEBUG="yes"
+
+## Type:       yesno
+## Default:    yes
+# All the messages go to syslog and stderr (script & C code)
+USE_SYSLOG="yes"
+
diff --git a/usr/etc/initd/initd.fcoe b/usr/etc/initd/initd.fcoe
old mode 100644
new mode 100755
index b8abbe1..44e34ff
--- a/usr/etc/initd/initd.fcoe
+++ b/usr/etc/initd/initd.fcoe
@@ -1,20 +1,20 @@
 #!/bin/sh
 #
-# Copyright(c) 2007 Intel Corporation. All rights reserved.
-# 
+# Copyright(c) 2009 Intel Corporation. All rights reserved.
+#
 # This program is free software; you can redistribute it and/or modify it
 # under the terms and conditions of the GNU General Public License,
 # version 2, as published by the Free Software Foundation.
-# 
+#
 # This program is distributed in the hope it will be useful, but WITHOUT
 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 # FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 # more details.
-# 
+#
 # You should have received a copy of the GNU General Public License along with
 # this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
-# 
+#
 # Maintained at www.Open-FCoE.org
 #
 #
@@ -54,7 +54,7 @@ fi
 # Switch on start / stop argument
 case "$1" in
 start)
-	
+
 	# start fcmon
 	lsmod | grep fcoe &> /dev/null
 	if [ "$?" -eq "1" ]
diff --git a/usr/etc/initd/initd.suse b/usr/etc/initd/initd.suse
new file mode 100755
index 0000000..a11cbca
--- /dev/null
+++ b/usr/etc/initd/initd.suse
@@ -0,0 +1,376 @@
+#!/bin/sh
+#
+# Copyright(c) 2009 Intel Corporation. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# Maintained at www.Open-FCoE.org
+#
+#
+# Open-FCoE User-Space Software
+# Based on:
+#     Template LSB system startup script for example service/daemon FOO
+#     Copyright (C) 1995--2005  Kurt Garloff, SUSE / Novell Inc.
+#
+# /etc/init.d/fcoe         This shell script takes care of starting and stopping
+#                          of the fcoemon daemon
+#   and its symbolic link
+# /sbin/rcfcoe
+#
+# chkconfig: 345 20 80
+#
+### BEGIN INIT INFO
+# Provides: fcoe
+# Required-Start: network
+# Required-Stop:
+# Default-Start: 3 5
+# Default-Stop: 3 5
+# Description: Open-FCoE SAN Setup
+### END INIT INFO
+
+CONFIG_DIR=/etc/fcoe
+CONFIG_SCRIPT=$CONFIG_DIR/scripts/fcoeplumb
+PID_FILE="/var/run/fcoemon.pid"
+LOG_FILE="/var/log/fcoemon.log"
+FCOEMON=/sbin/fcoemon
+FCOEADM=/sbin/fcoeadm
+DCBD=dcbd
+LOGGER="logger -t fcoe -s"
+
+. /etc/rc.status
+rc_reset
+
+test -r $CONFIG_DIR/config || {
+	$LOGGER "$CONFIG_DIR/config not installed";
+	if [ "$1" = "stop" ]; then exit 0;
+	else
+		rc_failed
+		rc_status -v
+		rc_exit
+	fi
+}
+
+varify_yesno_values()
+{
+	value=$1
+
+	case $value in
+		"yes" | "YES" ) return 1 ;;
+		"no" | "NO" ) return 2 ;;
+		*) return 0 ;;
+	esac
+}
+
+verify_configuration()
+{
+	. $CONFIG_DIR/config
+	varify_yesno_values $USE_SYSLOG
+	if [ $? -eq 0 ]; then
+		echo "Invalid USE_SYSLOG in $CONFIG_DIR/config"
+		return 1
+	fi
+	varify_yesno_values $DEBUG
+	if [ $? -eq 0 ]; then
+		echo "Invalid DEBUG in $CONFIG_DIR/config"
+		return 1
+	fi
+
+	for ifcfg_file in `ls $CONFIG_DIR/cfg-eth*`
+	do
+		. $ifcfg_file
+		varify_yesno_values $FCOE_ENABLE
+		if [ $? -eq 0 ]; then
+			echo "Invalid FCOE_ENABLE in $ifcfg_file"
+			return 1
+		fi
+		varify_yesno_values $DCB_REQUIRED
+		if [ $? -eq 0 ]; then
+			echo "Invalid DCB_REQUIRED in $ifcfg_file"
+			return 1
+		fi
+	done
+	return 0
+}
+
+verify_configuration
+if [ $? -ne 0 ]; then
+	rc_failed
+	rc_status -v
+	rc_exit
+fi
+if [ "$USE_SYSLOG" != "yes" ] && [ "$USE_SYSLOG" != "YES" ]; then
+	LOGGER="echo"
+fi
+if [ "$DEBUG" = "yes" ] || [ "$DEBUG" = "YES" ]; then
+	DEBUG="yes"
+fi
+
+test -x $CONFIG_SCRIPT || {
+	$LOGGER "$CONFIG_SCRIPT not installed";
+	if [ "$1" = "stop" ]; then exit 0;
+	else
+		rc_failed
+		rc_status -v
+		rc_exit
+	fi
+}
+
+test -x $FCOEADM || {
+	$LOGGER "$FCOEADM not installed";
+	if [ "$1" = "stop" ]; then exit 0;
+	else
+		rc_failed
+		rc_status -v
+		rc_exit
+	fi
+}
+
+test -x $FCOEMON || {
+	$LOGGER "$FCOEMON not installed";
+	if [ "$1" = "stop" ]; then exit 0;
+	else
+		rc_failed
+		rc_status -v
+		rc_exit
+	fi
+}
+
+startup_fcoe_modules()
+{
+	modprobe fcoe > /dev/null 2>&1
+}
+
+configure_dcb_interface()
+{
+	ifname=$1
+
+	checkproc $DCBD
+	if [ $? -ne 0 ]; then
+		MSGLINE1="Fatal: dcbd is not runnng.\n"
+		MSGLINE2="      The dcbd must be running because DCB is\n"
+		MSGLINE3="      configured as REQUIRED at Ethernet port $ifname."
+		MSG=`echo -e "$MSGLINE1$MSGLINE2$MSGLINE3"`
+		$LOGGER "$MSG"
+		return 1
+	fi
+
+	#
+	# At this moment, the dcbd may not be ready to
+	# accept any client connections.
+	#
+	retry_count=6
+	while true
+	do
+		[ $retry_count -eq 0 ] && break
+		retry_count=$(($retry_count-1))
+		dcbtool gc $ifname dcb > /dev/null 2>&1
+		if [ $? -ne 0 ]; then
+			usleep 500000
+			continue
+		fi
+		dcbtool sc $ifname dcb on > /dev/null 2>&1
+		dcbtool sc $ifname app:0 e:1 a:1 w:1 > /dev/null 2>&1
+		dcbtool sc $ifname ll:0 e:1 a:1 w:1 > /dev/null 2>&1
+		dcbtool sc $ifname pfc a:1 > /dev/null 2>&1
+		dcbtool sc $ifname pg a:0 > /dev/null 2>&1
+		STATUS=`dcbtool gc $ifname dcb 2>&1 | awk '/Status:/{print $2}'`
+		STATE=`dcbtool gc $ifname dcb 2>&1 | awk '/DCB State:/{print $3}'`
+		if [ "$STATUS" = "Successful" ] && [ "$STATE" = "on" ]; then
+			return 0
+		fi
+		# Not likely to reach here!
+		usleep 500000
+	done
+	$LOGGER "Warning: Failed to configure dcb at $ifname"
+	return 1
+}
+
+validate_link_flow_control()
+{
+	ifname=$1
+
+	retry_count=1
+	while true
+	do
+		TX_STATUS=`ethtool -a $ifname 2>&1 | awk '/TX:/{print $2}'`
+		RX_STATUS=`ethtool -a $ifname 2>&1 | awk '/RX:/{print $2}'`
+
+		if [ "$TX_STATUS" = "on" ] && [ "$RX_STATUS" = "on" ]; then
+			return 0
+		fi
+
+		ethtool -A $ifname rx on tx on
+		[ $retry_count -eq 0 ] && return 0
+		retry_count=$(($retry_count-1))
+		usleep 500000
+	done
+	$LOGGER "Warning: Failed to bring up link flow control of $ifname."
+	return 1
+}
+
+service_start()
+{
+	checkproc ${FCOEMON}
+	if [ $? -eq 0 ]; then
+		$LOGGER "Warning: daemon already running."
+		return
+	fi
+	rm -f /var/run/fcoemon.*
+
+	startup_fcoe_modules
+
+	HAS_DCB_IF="false"
+	for ifcfg_file in `ls $CONFIG_DIR/cfg-eth*`
+	do
+		ifname=`basename $ifcfg_file | cut -d"-" -f2`
+		. $ifcfg_file
+		[ "$FCOE_ENABLE" != "yes" ] &&
+		[ "$FCOE_ENABLE" != "YES" ] && continue
+
+		if [ "$DCB_REQUIRED" != "yes" ] &&
+		   [ "$DCB_REQUIRED" != "YES" ]; then
+			#
+			# DCB is not required, we nee to validate the
+			# link flow control of the Ethernet port
+			#
+			validate_link_flow_control $ifname
+			[ $? -ne 0 ] && continue
+			#
+			# Create the FCoE interface
+			#
+			$FCOEADM -c $ifname
+		else
+			#
+			# Configure the DCB for the Ethernet
+			# port if DCB is required
+			#
+			configure_dcb_interface $ifname
+			if [ $? -eq 0 ]; then
+				HAS_DCB_IF="true"
+			fi
+		fi
+	done
+
+	#
+	# If DCB-required Ethernet port exists, then start the fcoemon
+	# daemon to create the FCoE interfaces for these ports.
+	#
+	if [ "$HAS_DCB_IF" = "true" ]; then
+		startproc -l ${LOG_FILE} -p ${PID_FILE} ${FCOEMON}
+	fi
+
+	rc_status -v
+	return
+}
+
+service_stop()
+{
+	checkproc $FCOEMON
+	[ $? -eq 0 ] && killproc -TERM $FCOEMON
+
+	for ifcfg_file in `ls $CONFIG_DIR/cfg-eth*`
+	do
+		ifname=`basename $ifcfg_file | cut -d"-" -f2`
+		. $ifcfg_file
+		[ "$FCOE_ENABLE" != "yes" ] &&
+		[ "$FCOE_ENABLE" != "YES" ] && continue
+		if [ "$DCB_REQUIRED" != "yes" ] &&
+		   [ "$DCB_REQUIRED" != "YES" ]; then
+			STATUS=`$FCOEADM -i $ifname 2>&1 | \
+				awk '/Interface Name:/{print $3}'`
+			if [ "$STATUS" = "$ifname" ]; then
+				$FCOEADM -d $ifname
+			fi
+		fi
+	done
+
+	retry_count=5
+	while true
+	do
+		[ $retry_count -eq 0 ] && break
+		ps | grep -q fcoeplumb
+		[ $? -ne 0 ] && [ ! -f $PID_FILE ] && break
+		sleep 1
+		retry_count=$(($retry_count-1))
+	done
+	rm -f /var/run/fcoemon.*
+	rm -f /tmp/fcoemon.dcbd.*
+
+	rc_status -v
+}
+
+service_status()
+{
+	rc_status -v
+	checkproc $FCOEMON
+	if [ $? -eq 0 ]; then
+		echo "$FCOEMON -- RUNNING, pid=`cat $PID_FILE`"
+	else
+		echo "$FCOEMON -- UNUSED"
+	fi
+	IF_LIST=`$FCOEADM -i 2>&1 | \
+		awk '/Interface Name:/{print $3}' | \
+		sort | awk '{printf("%s ", $1)}'`
+	if [ -z "$IF_LIST" ]; then
+		echo "No interfaces created."
+	else
+		echo "Created interfaces: $IF_LIST"
+	fi
+}
+
+case "$1" in
+	start)
+		$LOGGER "Service starting up"
+		service_start
+		;;
+	stop)
+		$LOGGER "Service shutting down"
+		service_stop
+		;;
+	try-restart|condrestart)
+		if test "$1" = "condrestart"; then
+			$LOGGER "${attn} Use try-restart ${done}(LSB)${attn} " \
+				"rather than condrestart ${warn}(RH)${norm}"
+		fi
+		$0 status
+		if test $? = 0; then
+			$0 restart
+		else
+			rc_reset	# Not running is not a failure.
+		fi
+		rc_status
+		;;
+	restart)
+		$0 stop
+		$0 start
+		rc_status
+		;;
+	force-reload)
+		$0 try-restart
+		rc_status
+		;;
+	reload)
+		$LOGGER "Service reloading"
+		rc_failed 3
+		rc_status -v
+		;;
+	status)
+		service_status
+		;;
+	*)
+		echo "Usage: $0 {start|stop|status|try-restart|restart|force-reload|reload}"
+		exit 1
+		;;
+esac
+rc_exit
diff --git a/usr/tools/fcoeadm/Makefile b/usr/tools/fcoeadm/Makefile
index 98e6424..7994216 100644
--- a/usr/tools/fcoeadm/Makefile
+++ b/usr/tools/fcoeadm/Makefile
@@ -90,7 +90,7 @@ version.c: FORCE
 		Please read the INSTALL file.))
 	@echo '       BUILD $(@F)'
 	@$(RM) -f $@
-	@echo char tools_version[] = '"'$(BUILD_VERSION) $(shell date)'";' > $@
+	@echo char build_date[] = '"'$(shell date)'";' > $@
 
 .PHONY: FORCE
 
diff --git a/usr/tools/fcoeadm/fcoeadm.c b/usr/tools/fcoeadm/fcoeadm.c
index 3de9c2c..c98859a 100644
--- a/usr/tools/fcoeadm/fcoeadm.c
+++ b/usr/tools/fcoeadm/fcoeadm.c
@@ -1,5 +1,5 @@
 /*
- * Copyright(c) 2008 Intel Corporation. All rights reserved.
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms and conditions of the GNU General Public License,
@@ -19,12 +19,18 @@
 
 #include <stdio.h>
 #include <stdlib.h>
+#include <libgen.h>
 #include <errno.h>
 #include <getopt.h>
 #include <dirent.h>
 #include <string.h>
 #include "fcoeadm.h"
 
+static char *fcoeadm_version = "\
+fcoeadm v1.0.6\n\
+Copyright (c) 2009, Intel Corporation.\n\
+";
+
 #define SYSFS_MOUNT	"/sys"
 #define SYSFS_NET	SYSFS_MOUNT "/class/net"
 #define SYSFS_FCHOST	SYSFS_MOUNT "/class/fc_host"
@@ -43,16 +49,17 @@ static struct option fcoeadm_opts[] = {
     {"lun", 1, 0, 'l'},
     {"stats", 1, 0, 's'},
     {"help", 0, 0, 'h'},
+    {"version", 0, 0, 'v'},
     {0, 0, 0, 0}
 };
 
 struct opt_info _opt_info, *opt_info = &_opt_info;
-char progname[256];
+char progname[20];
 
 static void
 fcoeadm_help(void)
 {
-	/* printf("Build Date: %s\n", BUILD_DATE); */
+	printf("%s\n", fcoeadm_version);
 	printf("Usage: %s\n"
 		"\t [-c|--create] <ethX>\n"
 		"\t [-d|--destroy] <ethX>\n"
@@ -61,7 +68,8 @@ fcoeadm_help(void)
 		"\t [-t|--target] [<ethX>]\n"
 		"\t [-l|--lun] [<target port_id> [<lun_id>]]\n"
 		"\t [-s|--stats] <ethX> [-n <interval>]\n"
-		"\t [-h|--help]\n", progname);
+		"\t [-v|--version]\n"
+		"\t [-h|--help]\n\n", progname);
 }
 
 /*
@@ -225,7 +233,7 @@ fcoeadm_find_fchost(char *ifname, char *fchost, int len)
 						  namelist[n]->d_name)) {
 				dname_len = strnlen(namelist[n]->d_name, len);
 				if (dname_len != len) {
-					/* 
+					/*
 					 * This assumes that d_name is always
 					 * NULL terminated.
 					 */
@@ -432,10 +440,10 @@ int main(int argc, char *argv[])
 		exit(-EINVAL);
 	}
 
-	strncpy(progname, argv[0], sizeof(progname));
+	strncpy(progname, basename(argv[0]), sizeof(progname));
 	memset(opt_info, 0, sizeof(*opt_info));
 
-	while ((opt = getopt_long(argc, argv, "c:d:r:itls:n:h",
+	while ((opt = getopt_long(argc, argv, "c:d:r:itls:n:hv",
 				  fcoeadm_opts, NULL)) != -1) {
 		switch (opt) {
 		case 'c':
@@ -525,6 +533,9 @@ int main(int argc, char *argv[])
 			if (!opt_info->s_flag)
 				goto error;
 			goto stats;
+		case 'v':
+			printf("%s\n", fcoeadm_version);
+			goto done;
 		case 'h':
 		default:
 			fcoeadm_help();
diff --git a/usr/tools/fcoeadm/fcoeadm.h b/usr/tools/fcoeadm/fcoeadm.h
index 7b2cb37..196812c 100644
--- a/usr/tools/fcoeadm/fcoeadm.h
+++ b/usr/tools/fcoeadm/fcoeadm.h
@@ -1,5 +1,5 @@
 /*
- * Copyright(c) 2008 Intel Corporation. All rights reserved.
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms and conditions of the GNU General Public License,
@@ -17,6 +17,9 @@
  * Maintained at www.Open-FCoE.org
  */
 
+#ifndef _FCOEADM_H_
+#define _FCOEADM_H_
+
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/param.h>
@@ -60,7 +63,10 @@ struct opt_info {
 	int n_interval;		/* seconds */
 };
 extern struct opt_info *opt_info;
+extern char build_date[];
 
 extern void display_adapter_info(struct opt_info *opt_info);
 extern void display_target_info(struct opt_info *opt_info);
 extern void display_port_stats(struct opt_info *opt_info);
+
+#endif /* _FCOEADM_H_ */
diff --git a/usr/tools/fcoeadm/fcoeadm_display.c b/usr/tools/fcoeadm/fcoeadm_display.c
index 2f4c400..43d7d4c 100644
--- a/usr/tools/fcoeadm/fcoeadm_display.c
+++ b/usr/tools/fcoeadm/fcoeadm_display.c
@@ -1,5 +1,5 @@
 /*
- * Copyright(c) 2008 Intel Corporation. All rights reserved.
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms and conditions of the GNU General Public License,
diff --git a/usr/tools/fcoemon/Makefile b/usr/tools/fcoemon/Makefile
new file mode 100644
index 0000000..a0fe7e6
--- /dev/null
+++ b/usr/tools/fcoemon/Makefile
@@ -0,0 +1,155 @@
+# File: open-fcoe/usr/tools/fcoemon/Makefile
+
+# set the following corresepondingly to your preferred locations
+DESTDIR ?=
+
+#
+# Installation directory of the tools
+#
+ETCDIR = /etc
+EXEC_PREFIX = /
+SBINDIR = $(DESTDIR)$(EXEC_PREFIX)sbin
+PREFIX = /usr/
+MANDIR = $(DESTDIR)$(PREFIX)share/man
+
+SERVICE_NAME=fcoe
+SERVICE_DIR=$(DESTDIR)/etc/init.d
+CONFIG_DIR=$(DESTDIR)/etc/$(SERVICE_NAME)
+
+# Source directory of init.d
+INITD_SRC=../../etc/initd
+
+#
+# List of legal build component names
+#
+LEGAL_ARCH = i386 i486 i586 i686 x86_64
+LEGAL_OS = linux Linux
+
+#
+#
+# Validating OS and the machine architecture.
+#
+ifneq "$(filter-out $(LEGAL_ARCH), $(shell uname -i))" ""
+    $(error bad build architecture $(shell uname -i))
+else
+ifneq "$(filter-out $(LEGAL_OS), $(shell uname -s))" ""
+    $(error bad build OS $(shell uname -s))
+else
+
+CC = gcc
+LD = gcc
+CAT = cat
+AWK = awk
+SORT = sort
+INSTALL = install
+RM = rm
+GZIP = gzip
+CHKCONFIG = chkconfig
+LN = ln
+
+IFNAMES=$(shell $(CAT) /proc/net/dev | \
+		$(AWK) -F':' '/eth/ {print $$1}' | \
+		$(SORT) -u)
+
+OSNAME = $(shell uname -s)
+MANPAGES = fcoemon.8
+
+CFLAGS += -fPIC -O0 -g -Wall -Werror
+CFLAGS += -D$(OSNAME) -D_GNU_SOURCE
+CFLAGS += -I.
+CFLAGS += -I$(LIBHBALINUX)/include
+CFLAGS += -I$(DCB_SRC)/include
+
+ifeq ($(shell uname -i),x86_64)
+	CFLAGS += -m64
+else
+	CFLAGS += -m32
+	LDFLAGS += -melf_i386
+endif
+
+PROGRAMS = fcoemon
+
+default: all
+
+all: $(PROGRAMS)
+
+clean:
+	@$(RM) -f *.o version.* $(PROGRAMS) > /dev/null 2>&1
+
+.c.o:
+	@echo '       CC PIC' $<
+	@$(CC) $(CFLAGS) -c -o $@ $<
+
+fcoemon: version.o fcoemon_utils.o fcoemon.o
+	@echo '       LINK' $@
+	@$(LD) -o $@ $(LDFLAGS) -lrt $^
+
+install: all install_config install_doc
+	@$(if $(PROGRAMS), $(foreach i, $(PROGRAMS), \
+		$(INSTALL) -v -m 744 $(i) $(SBINDIR) ; ))
+
+install_config:
+	@$(INSTALL) -d $(CONFIG_DIR)
+	@if [ ! -f $(CONFIG_DIR)/config ]; then \
+		$(INSTALL) -v -m 644 $(INITD_SRC)/config \
+			$(CONFIG_DIR)/config ; \
+	fi ;
+	@$(INSTALL) -d $(CONFIG_DIR)/scripts
+	@$(INSTALL) -v -m 755 fcoeplumb.sh \
+		$(CONFIG_DIR)/scripts/fcoeplumb
+	@$(foreach i, $(IFNAMES), \
+		if [ ! -f $(CONFIG_DIR)/cfg-$(i) ]; then \
+			$(INSTALL) -v -m 744 $(INITD_SRC)/cfg-ethx \
+				$(CONFIG_DIR)/cfg-$(i) ; \
+		fi ; \
+	)
+	@if [ -f /etc/fedora-release ]; then \
+		$(INSTALL) -v -m 744 $(INITD_SRC)/initd.fcoe \
+			$(SERVICE_DIR)/$(SERVICE_NAME) ; \
+	elif [ -f /etc/SuSE-release ]; then \
+		$(INSTALL) -v -m 744 $(INITD_SRC)/initd.suse \
+			$(SERVICE_DIR)/$(SERVICE_NAME) ; \
+		$(RM) -f $(SBINDIR)/rc$(SERVICE_NAME) ; \
+		$(LN) -s $(SERVICE_DIR)/$(SERVICE_NAME) \
+			$(SBINDIR)/rc$(SERVICE_NAME); \
+	else \
+		$(INSTALL) -v -m 744 $(INITD_SRC)/initd.fcoe \
+			$(SERVICE_DIR)/$(SERVICE_NAME) ; \
+	fi
+	@$(CHKCONFIG) $(SERVICE_NAME) on
+
+install_doc:
+	@$(INSTALL) -d $(MANDIR)/man8
+	@$(if $(MANPAGES), $(foreach i, $(MANPAGES), \
+		$(INSTALL) -v -m 644 ../../doc/$(i) \
+			$(MANDIR)/man8 ; \
+		$(RM) -f $(MANDIR)/man8/$(i).gz ; \
+		$(GZIP) -9 $(MANDIR)/man8/$(i) ; ))
+
+uninstall: uninstall_config uninstall_doc
+	@$(if $(PROGRAMS), $(foreach i, $(PROGRAMS), \
+		$(RM) -vf $(SBINDIR)/$(i) ; ))
+
+uninstall_config:
+	@$(CHKCONFIG) $(SERVICE_NAME) off
+	@$(RM) -v -rf $(CONFIG_DIR)
+	@$(RM) -v -f $(SERVICE_DIR)/$(SERVICE_NAME)
+	@$(RM) -v -f $(SBINDIR)/rc$(SERVICE_NAME)
+
+uninstall_doc:
+	@$(if $(MANPAGES), $(foreach i, $(MANPAGES), \
+		$(RM) -vf $(MANDIR)/man8/$(i).gz ; ))
+
+version.c: FORCE
+	@$(if $(DCB_SRC),, $(error Error! DCB_SRC is not defined. \
+		Please read the INSTALL file.))
+	@$(if $(LIBHBALINUX),, $(error Error! LIBHBALINUX is not defined. \
+		Please read the INSTALL file.))
+	@echo '       BUILD $(@F)'
+	@$(RM) -f $@
+	@echo char build_date[] = '"'$(shell date)'";' > $@
+
+.PHONY: FORCE
+
+endif
+endif
diff --git a/usr/tools/fcoemon/fcoemon.c b/usr/tools/fcoemon/fcoemon.c
new file mode 100644
index 0000000..adaa581
--- /dev/null
+++ b/usr/tools/fcoemon/fcoemon.c
@@ -0,0 +1,2042 @@
+/*
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Maintained at www.Open-FCoE.org
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <getopt.h>
+#include <malloc.h>
+#include <signal.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
+#include <time.h>
+#include <libgen.h>
+#include <ulimit.h>
+#include <unistd.h>
+#include <paths.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/queue.h>
+#include <sys/un.h>
+#include <linux/sockios.h>
+#include <linux/if.h>
+#include <linux/if_arp.h>
+#include <linux/netlink.h>
+#include <linux/rtnetlink.h>
+#include <linux/ethtool.h>
+
+#include "net_types.h"
+#include "fc_types.h"
+
+#include "fcoemon_utils.h"
+#include "fcoemon.h"
+
+static char *fcoemon_version = "\
+fcoemon v1.0.6\n\
+Copyright (c) 2009, Intel Corporation.\n\
+";
+
+/*
+ * Defines for FCoE config file.
+ */
+#define CONFIG_DIR                  "/etc/fcoe"
+#define CONFIG_MIN_VAL_LEN          (1 + 2)
+#define CONFIG_MAX_VAL_LEN          (20 + 2)
+#define DCB_APP_0_DEFAULT_ENABLE    1
+#define DCB_APP_0_DEFAULT_WILLING   1
+#define FCM_DEFAULT_QOS_MASK        (1 << 3)
+#define FILE_NAME_LEN               (NAME_MAX + 1)
+
+/*
+ * fcoe service configuration data
+ * Note: These information are read in from the fcoe service
+ *       files in CONFIG_DIR
+ */
+struct fcoe_port_config {
+	struct fcoe_port_config *next;
+	char ifname[IFNAMSIZ];
+	int fcoe_enable;
+	int has_fip;
+	int dcb_required;
+	int dcb_app_0_enable;
+	int dcb_app_0_willing;
+};
+
+struct fcoe_config {
+	int debug;
+	int use_syslog;
+	struct fcoe_port_config *port;
+} fcoe_config;
+
+static u_int8_t fcm_def_qos_mask = FCM_DEFAULT_QOS_MASK;
+
+struct clif;			/* for dcbtool.h only */
+#include "dcb_types.h"
+#include "dcbtool.h"		/* for typedef cmd_status */
+#include "clif.h"
+#include "clif_cmds.h"
+#include "common.h"		/* for event msg level definitions */
+
+#define CLIF_NAME_PATH          _PATH_VARRUN "dcbd/clif"
+#define CLIF_PID_FILE           _PATH_VARRUN "fcoemon.pid"
+#define CLIF_LOCAL_SUN_PATH     _PATH_TMP "fcoemon.dcbd.%d"
+#define FCM_DCBD_TIMEOUT_USEC   (30 * 1000 * 1000)	/* 30 seconds */
+#define FCM_EVENT_TIMEOUT_USEC  (500 * 1000)		/* half a second */
+
+/*
+ * Interact with DCB daemon.
+ */
+static void fcm_event_timeout(void *);
+static void fcm_dcbd_timeout(void *);
+static void fcm_dcbd_disconnect(void);
+static void fcm_dcbd_request(char *);
+static void fcm_dcbd_rx(void *);
+static void fcm_dcbd_ex(void *);
+static void fcm_dcbd_next(void);
+static void fcm_dcbd_event(char *, size_t);
+static void fcm_dcbd_cmd_resp(char *, cmd_status);
+static void fcm_dcbd_port_advance(struct fcm_fcoe *);
+static void fcm_dcbd_setup(struct fcm_fcoe *, u_int);
+
+struct fcm_clif {
+	int cl_fd;
+	int cl_busy;		/* non-zero if command pending */
+	int cl_ping_pending;
+	struct sockaddr_un cl_local;
+};
+
+static struct fcm_clif fcm_clif_st;
+static struct fcm_clif *fcm_clif = &fcm_clif_st;
+static struct sa_timer fcm_dcbd_timer;
+
+char *fcm_dcbd_cmd = CONFIG_DIR "/scripts/fcoeplumb";
+int fcm_dcbd_debug;
+int fcm_link_debug;
+int fcm_use_syslog;
+
+/* Debugging routine */
+static void print_errors(char *buf, int errors);
+
+struct fcm_fcoe_head fcm_fcoe_head = TAILQ_HEAD_INITIALIZER(fcm_fcoe_head);
+
+static int fcm_link_socket;
+static int fcm_link_seq;
+static void fcm_link_recv(void *);
+static void fcm_link_getlink(void);
+static int fcm_link_buf_check(size_t);
+
+/*
+ * Table for getopt_long(3).
+ */
+static struct option fcm_options[] = {
+	{"debug", 0, NULL, 'd'},
+	{"exec", 1, NULL, 'e'},
+	{"version", 0, NULL, 'v'},
+	{NULL, 0, NULL, 0}
+};
+
+char progname[20];
+
+static char fcm_pidfile[] = CLIF_PID_FILE;
+
+/*
+ * Issue with buffer size:  It isn't clear how to read more than one
+ * buffer's worth of GETLINK replies.  The kernel seems to just drop the
+ * interface messages if they don't fit in the buffer, so we just make it
+ * large enough to fit and expand it if we ever do a read that almost fills it.
+ */
+static char *fcm_link_buf;
+static size_t fcm_link_buf_size = 4096;	/* initial size */
+static const size_t fcm_link_buf_fuzz = 300;	/* "almost full" remainder */
+
+/*
+ * A value must be surrounded by quates, e.g. "x".
+ * The minimum length of a value is 1 excluding the quotes.
+ * The maximum length of a value is 20 excluding the quotes.
+ */
+static int
+fcm_remove_quotes(char *buf, int len)
+{
+	char *s = buf;
+	char *e = buf + len - 1;
+	char tmp[CONFIG_MAX_VAL_LEN + 1];
+
+	if (strnlen(buf, len) < CONFIG_MIN_VAL_LEN)
+		return -1;
+	if ((*s >= '0' && *s <= '9') ||
+	    (*s >= 'a' && *s <= 'z') ||
+	    (*s >= 'A' && *s <= 'Z'))
+		return -1;
+	if ((*e >= '0' && *e <= '9') ||
+	    (*e >= 'a' && *e <= 'z') ||
+	    (*e >= 'A' && *e <= 'Z'))
+		return -1;
+	s = buf + 1;
+	*e = '\0';
+	strncpy(tmp, s, len - 1);
+	strncpy(buf, tmp, len - 1);
+
+	return 0;
+}
+
+/*
+ * Read a configuration variable for a port from a config file.
+ * There's no problem if the file doesn't exist.
+ * The buffer is set to an empty string if the variable is not found.
+ *
+ * Returns:  1    found
+ *           0    not found
+ *           -1   error in format
+ */
+static size_t
+fcm_read_config_variable(char *file, char *val_buf, size_t len,
+			FILE *fp, const char *var_name)
+{
+	char *s;
+	char *var;
+	char *val;
+	char buf[FILE_NAME_LEN];
+	int n;
+
+	val_buf[0] = '\0';
+	buf[sizeof(buf) - 1] = '\0';
+	while ((s = fgets(buf, sizeof(buf) - 1, fp)) != NULL) {
+		while (isspace(*s))
+			s++;
+		if (*s == '\0' || *s == '#')
+			continue;
+		var = s;
+		if (!isalpha(*var))
+			continue;
+		val = strchr(s, '=');
+		if (val == NULL)
+			continue;
+		*val++ = '\0';
+		s = val;
+		if (strcmp(var_name, var) != 0)
+			continue;
+		while (*s != '\0' && !isspace(*s))
+			s++;
+		*s = '\0';
+		n = snprintf(val_buf, len, "%s", val);
+		if (fcm_remove_quotes(val_buf, n) < 0) {
+			SA_LOG("Invalid format in config file"
+				" %s: %s=%s\n",
+				file, var_name, val);
+			/* error */
+			return -1;
+		}
+		/* found */
+		return 1;
+	}
+	/* not found */
+	return 0;
+}
+
+static int
+fcm_read_config_files(void)
+{
+	char file[80];
+	FILE *fp;
+	char val[CONFIG_MAX_VAL_LEN + 1];
+	DIR *dir;
+	struct dirent *dp;
+	struct fcoe_port_config *curr;
+	struct fcoe_port_config *next;
+	int rc;
+
+	memset(&fcoe_config, 0, sizeof(fcoe_config));
+
+	strncpy(file, CONFIG_DIR "/" "config", sizeof(file));
+	fp = fopen(file, "r");
+	if (!fp) {
+		SA_LOG_ERR(errno, "Failed reading %s\n", file);
+		exit(1);
+	}
+
+	rc = fcm_read_config_variable(file, val,
+					sizeof(val), fp, "DEBUG");
+	if (rc < 0) {
+		fclose(fp);
+		return -1;
+	}
+	/* if not found, default to "yes" */
+	if (!strncasecmp(val, "yes", 3) || !rc) {
+		fcoe_config.debug = 1;
+		fcm_dcbd_debug = 1;
+		fcm_link_debug = 1;
+	}
+
+	rc = fcm_read_config_variable(file, val,
+					sizeof(val), fp, "USE_SYSLOG");
+	if (rc < 0) {
+		fclose(fp);
+		return -1;
+	}
+	/* if not found, default to "yes" */
+	if (!strncasecmp(val, "yes", 3) || !rc) {
+		fcoe_config.use_syslog = 1;
+		fcm_use_syslog = 1;
+	}
+
+	fclose(fp);
+
+	dir = opendir(CONFIG_DIR);
+	if (dir == NULL) {
+		SA_LOG_ERR(errno,
+			"Failed reading directory %s\n", CONFIG_DIR);
+		return -1;
+	}
+	for (;;) {
+		dp = readdir(dir);
+		if (dp == NULL)
+			break;
+		if (dp->d_name[0] == '.' && (dp->d_name[1] == '\0' ||
+		   (dp->d_name[1] == '.' && dp->d_name[2] == '\0')))
+			continue;
+		rc = strncmp(dp->d_name, "cfg-eth", strlen("cfg-eth"));
+		if (rc)
+			continue;
+		next = (struct fcoe_port_config *)
+				calloc(1, sizeof(struct fcoe_port_config));
+		if (!fcoe_config.port) {
+			fcoe_config.port = next;
+			curr = next;
+		} else {
+			curr->next = next;
+			curr = next;
+		}
+		strncpy(curr->ifname, dp->d_name + 4, sizeof(curr->ifname));
+		strncpy(file, CONFIG_DIR "/", sizeof(file));
+		strncat(file, dp->d_name, sizeof(file));
+		fp = fopen(file, "r");
+		if (!fp) {
+			SA_LOG_ERR(errno, "Failed reading %s\n", file);
+			exit(1);
+		}
+
+		/* FCOE_ENABLE */
+		rc = fcm_read_config_variable(file, val, sizeof(val),
+					fp, "FCOE_ENABLE");
+		if (rc < 0) {
+			fclose(fp);
+			return -1;
+		}
+		/* if not found, default to "no" */
+		if (!strncasecmp(val, "yes", 3) && rc == 1)
+			curr->fcoe_enable = 1;
+
+		/* DCB_REQUIRED */
+		rc = fcm_read_config_variable(file, val, sizeof(val),
+					fp, "DCB_REQUIRED");
+		if (rc < 0) {
+			fclose(fp);
+			return -1;
+		}
+		/* if not found, default to "no" */
+		if (!strncasecmp(val, "yes", 3) && rc == 1) {
+			curr->dcb_required = 1;
+			curr->dcb_app_0_enable = DCB_APP_0_DEFAULT_ENABLE;
+			curr->dcb_app_0_willing = DCB_APP_0_DEFAULT_WILLING;
+		}
+
+		/* HAS_FIP */
+		rc = fcm_read_config_variable(file, val,
+						sizeof(val), fp, "HAS_FIP");
+		if (rc < 0) {
+			fclose(fp);
+			return -1;
+		}
+		/* if not found, default to "no" */
+		if (!strncasecmp(val, "yes", 3) && rc == 1)
+			curr->has_fip = 1;
+
+		fclose(fp);
+	}
+	closedir(dir);
+	return 0;
+}
+
+static struct fcoe_port_config *
+fcm_find_port_config(char *ifname)
+{
+	struct fcoe_port_config *p;
+
+	p = fcoe_config.port;
+	while (p) {
+		if (!strncmp(ifname, p->ifname, IFNAMSIZ) &&
+		    p->fcoe_enable && p->dcb_required)
+			return p;
+		p = p->next;
+	}
+	return NULL;
+}
+
+static int
+fcm_link_init(void)
+{
+	int fd;
+	int rc;
+	struct sockaddr_nl l_local;
+
+	fcm_link_buf = malloc(fcm_link_buf_size);
+	ASSERT(fcm_link_buf);
+
+	fd = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE);
+	if (fd < 0) {
+		SA_LOG_ERR(errno, "socket error");
+		return fd;
+	}
+	memset(&l_local, 0, sizeof(l_local));
+	l_local.nl_family = AF_NETLINK;
+	l_local.nl_groups = RTMGRP_LINK;
+	l_local.nl_pid = 0;
+	rc = bind(fd, (struct sockaddr *)&l_local, sizeof(l_local));
+	if (rc == -1) {
+		SA_LOG_ERR(errno, "bind error");
+		return rc;
+	}
+	fcm_link_socket = fd;
+
+	/* Add a given file descriptor from a readfds set */
+	sa_select_add_fd(fd, fcm_link_recv, NULL, NULL, NULL);
+
+	fcm_link_getlink();
+
+	return 0;
+}
+
+static void
+fcm_link_recv(void *arg)
+{
+	int rc;
+	char *buf;
+	struct nlmsghdr *hp;
+	struct ifinfomsg *ip;
+	struct rtattr *ap;
+	struct fcm_fcoe *ff;
+	unsigned type;
+	int plen;
+	int rlen;
+	char ifname[IFNAMSIZ];
+
+	buf = fcm_link_buf;
+	rc = read(fcm_link_socket, buf, fcm_link_buf_size);
+	if (rc <= 0) {
+		if (rc < 0)
+			SA_LOG_ERR(errno, "read error");
+		return;
+	}
+	hp = (struct nlmsghdr *)buf;
+	rlen = rc;
+	for (hp = (struct nlmsghdr *)buf; NLMSG_OK(hp, rlen);
+	     hp = NLMSG_NEXT(hp, rlen)) {
+		type = hp->nlmsg_type;
+		if (hp->nlmsg_type == NLMSG_DONE)
+			break;
+		if (hp->nlmsg_type == NLMSG_ERROR) {
+			SA_LOG("nlmsg error");
+			break;
+		}
+		plen = NLMSG_PAYLOAD(hp, 0);
+		ip = (struct ifinfomsg *)NLMSG_DATA(hp);
+		if (plen < sizeof(*ip)) {
+			SA_LOG("too short (%d) to be a LINK message", rc);
+			break;
+		}
+		switch (type) {
+		case RTM_NEWLINK:
+			if (fcm_link_debug)
+				SA_LOG("newlink %d", ip->ifi_index);
+			goto rest;
+		case RTM_DELLINK:
+			if (fcm_link_debug)
+				SA_LOG("dellink %d", ip->ifi_index);
+			goto rest;
+		case RTM_GETLINK:
+			if (fcm_link_debug)
+				SA_LOG("getlink %d", ip->ifi_index);
+rest:
+			if (fcm_link_buf_check(rc)) {
+				fcm_link_getlink();
+				return;
+			}
+			if (ip->ifi_type != ARPHRD_ETHER)
+				break;
+
+			ff = fcm_fcoe_lookup_create_ifindex(ip->ifi_index);
+			if (ff == NULL)
+				break;
+			ff->ff_flags = ip->ifi_flags;
+
+			plen -= sizeof(*ip);
+			for (ap = (struct rtattr *)(ip + 1); RTA_OK(ap, plen);
+			     ap = RTA_NEXT(ap, plen)) {
+				switch (ap->rta_type) {
+				case IFLA_ADDRESS:
+					if (RTA_PAYLOAD(ap) == 6)
+						ff->ff_mac =
+							net48_get(RTA_DATA(ap));
+					break;
+				case IFLA_IFNAME:
+					sa_strncpy_safe(ifname, sizeof(ifname),
+							RTA_DATA(ap),
+							RTA_PAYLOAD(ap));
+					if (fcm_link_debug)
+						SA_LOG("ifname %s", ifname);
+					fcm_fcoe_set_name(ff, ifname);
+					break;
+				case IFLA_OPERSTATE:
+					ff->ff_operstate =
+					    *(uint8_t *) RTA_DATA(ap);
+					break;
+				default:
+					break;
+				}
+			}
+			break;
+		default:
+			break;
+		}
+	}
+	fcm_dcbd_update();
+}
+
+/*
+ * Send rt_netlink request for all network interfaces.
+ */
+static void
+fcm_link_getlink(void)
+{
+	struct {
+		struct nlmsghdr nl;
+		struct ifinfomsg ifi;	/* link level specific information,
+					   not dependent on network protocol */
+	} msg;
+	int rc;
+
+	memset(&msg, 0, sizeof(msg));
+	msg.nl.nlmsg_len = sizeof(msg);
+	msg.nl.nlmsg_type = RTM_GETLINK;
+	msg.nl.nlmsg_flags = NLM_F_REQUEST | NLM_F_ROOT | NLM_F_ATOMIC;
+	msg.nl.nlmsg_seq = ++fcm_link_seq;
+	/* msg.nl.nlmsg_pid = getpid(); */
+	msg.ifi.ifi_family = AF_UNSPEC;
+	msg.ifi.ifi_type = ARPHRD_ETHER;
+	rc = write(fcm_link_socket, &msg, sizeof(msg));
+	if (rc < 0)
+		SA_LOG_ERR(errno, "write error");
+}
+
+/*
+ * Check for whether buffer needs to grow based on amount read.
+ * Free's the old buffer so don't use that after this returns non-zero.
+ */
+static int
+fcm_link_buf_check(size_t read_len)
+{
+	char *buf;
+	size_t len = read_len;
+
+	if (len > fcm_link_buf_size - fcm_link_buf_fuzz) {
+		len = fcm_link_buf_size;
+		len = len + len / 2;	/* grow by 50% */
+		buf = malloc(len);
+		if (buf != NULL) {
+			free(fcm_link_buf);
+			fcm_link_buf = buf;
+			fcm_link_buf_size = len;
+			return 1;
+		}
+	}
+	return 0;
+}
+
+static void
+fcm_fcoe_init(void)
+{
+	if (fcm_read_config_files())
+		exit(1);
+}
+
+/*
+ * Allocate an FCoE interface state structure.
+ */
+static struct fcm_fcoe *
+fcm_fcoe_alloc(void)
+{
+	struct fcm_fcoe *ff;
+
+	ff = calloc(1, sizeof(*ff));
+	if (ff) {
+		ff->ff_qos_mask = fcm_def_qos_mask;
+		ff->ff_ifindex = ~0;
+		ff->ff_operstate = IF_OPER_UNKNOWN;
+		TAILQ_INSERT_TAIL(&fcm_fcoe_head, ff, ff_list);
+	}
+	return ff;
+}
+
+/*
+ * Find an FCoE interface by ifindex.
+ */
+static struct fcm_fcoe *
+fcm_fcoe_lookup_create_ifindex(u_int32_t ifindex)
+{
+	struct fcm_fcoe *ff;
+
+	TAILQ_FOREACH(ff, &fcm_fcoe_head, ff_list) {
+		if (ff->ff_ifindex == ifindex)
+			return ff;
+	}
+	ff = fcm_fcoe_alloc();
+	if (ff != NULL) {
+		ff->ff_ifindex = ifindex;
+		ff->ff_pfc_saved.u.pfcup = 0xffff;
+		sa_timer_init(&ff->ff_event_timer, fcm_event_timeout, ff);
+	}
+	return ff;
+}
+
+/*
+ * Find an FCoE interface by name.
+ * What about VLAN instances?  They can't use DCB, perhaps.
+ */
+static struct fcm_fcoe *
+fcm_fcoe_lookup_name(char *name)
+{
+	struct fcm_fcoe *ff;
+
+	TAILQ_FOREACH(ff, &fcm_fcoe_head, ff_list) {
+		if (strcmp(ff->ff_name, name) == 0)
+			break;
+	}
+	return ff;
+}
+
+#ifdef NOT_YET
+/*
+ * Find an FCoE interface by MAC address.
+ */
+static struct fcm_fcoe *
+fcm_fcoe_lookup_mac(u_int64_t mac, int vlan)
+{
+	struct fcm_fcoe *ff;
+
+	TAILQ_FOREACH(ff, &fcm_fcoe_head, ff_list) {
+		if (ff->ff_mac == mac && ff->ff_vlan == vlan)
+			break;
+	}
+	return ff;
+}
+
+/*
+ * Find or create an FCoE interface by MAC address and VLAN ID.
+ * If vlan is -1, find the base interface.
+ */
+static struct fcm_fcoe *
+fcm_fcoe_lookup_create_mac(u_int64_t mac, int vlan)
+{
+	struct fcm_fcoe *ff;
+
+	ff = fcm_fcoe_lookup_mac(mac, vlan);
+	if (ff == NULL) {
+		ff = fcm_fcoe_alloc();
+		if (ff != NULL) {
+			ff->ff_mac = mac;
+			ff->ff_vlan = vlan;
+		}
+	}
+	return ff;
+}
+#endif
+
+static void
+fcm_fcoe_get_dcb_settings(struct fcm_fcoe *ff)
+{
+	fc_wwn_t wwpn;
+	int vlan = ff->ff_vlan;
+	struct fcoe_port_config *p;
+
+	if (ff->ff_mac == 0)
+		return;		/* loopback or other non-eligible interface */
+
+	/*
+	 * Get DCB config from file if possible.
+	 */
+	wwpn = fc_wwn_from_mac(ff->ff_mac, 2, vlan >= 0 ? vlan : 0);
+
+	p = fcoe_config.port;
+	while (p) {
+		if (!strncmp(ff->ff_name, p->ifname, IFNAMSIZ)) {
+			ff->ff_app_info.enable = p->dcb_app_0_enable;
+			ff->ff_app_info.willing = p->dcb_app_0_willing;
+			ff->ff_has_fip = p->has_fip;
+			break;
+		}
+		p = p->next;
+	}
+}
+
+static void
+fcm_fcoe_set_name(struct fcm_fcoe *ff, char *ifname)
+{
+	char *cp;
+	int vlan;
+
+	snprintf(ff->ff_name, sizeof(ff->ff_name), "%s", ifname);
+	vlan = -1;
+	cp = strchr(ff->ff_name, '.');
+	if (cp != NULL) {
+		vlan = atoi(cp + 1);
+		if (vlan < 0 || vlan > 4095)
+			vlan = 0;
+	}
+	ff->ff_vlan = vlan;
+}
+
+static int
+fcm_fcoe_port_ready(struct fcm_fcoe *ff)
+{
+	int rc;
+
+	rc = (ff->ff_flags & (IFF_UP|IFF_RUNNING)) == (IFF_UP|IFF_RUNNING) &&
+	     ff->ff_operstate == IF_OPER_UP;
+
+	return rc;
+}
+
+static void
+fcm_dcbd_init()
+{
+	fcm_clif->cl_fd = -1;	/* not connected */
+	fcm_clif->cl_ping_pending = 0;
+	sa_timer_init(&fcm_dcbd_timer, fcm_dcbd_timeout, NULL);
+	fcm_dcbd_timeout(NULL);
+}
+
+static void
+fcm_dcbd_connect(void)
+{
+	int rc;
+	int fd;
+	struct sockaddr_un dest;
+	struct sockaddr_un *lp;
+
+	ASSERT(fcm_clif->cl_fd < 0);
+	fd = socket(PF_UNIX, SOCK_DGRAM, 0);
+	if (fd < 0) {
+		SA_LOG_ERR(errno, "clif socket open failed");	/* XXX */
+		return;
+	}
+
+	lp = &fcm_clif->cl_local;
+	lp->sun_family = PF_UNIX;
+	snprintf(lp->sun_path, sizeof(lp->sun_path),
+		CLIF_LOCAL_SUN_PATH, getpid());
+	rc = bind(fd, (struct sockaddr *)lp, sizeof(*lp));
+	if (rc < 0) {
+		SA_LOG_ERR(errno, "clif bind failed");
+		close(fd);
+		return;
+	}
+
+	memset(&dest, 0, sizeof(dest));
+	dest.sun_family = PF_UNIX;
+	snprintf(dest.sun_path, sizeof(dest.sun_path),
+		CLIF_NAME_PATH);
+	rc = connect(fd, (struct sockaddr *)&dest, sizeof(dest));
+	if (rc < 0) {
+		SA_LOG_ERR(errno, "clif connect failed");
+		unlink(lp->sun_path);
+		close(fd);
+		return;
+	}
+	fcm_clif->cl_fd = fd;
+	sa_select_add_fd(fd, fcm_dcbd_rx, NULL, fcm_dcbd_ex, fcm_clif);
+	if (fcm_dcbd_debug)
+		SA_LOG("connected to dcbd");
+}
+
+static int
+is_query_in_progress(void)
+{
+	struct fcm_fcoe *ff;
+
+	TAILQ_FOREACH(ff, &fcm_fcoe_head, ff_list) {
+		if (ff->ff_dcbd_state >= FCD_GET_PFC_CONFIG &&
+		    ff->ff_dcbd_state < FCD_DONE)
+			return 1;
+	}
+	return 0;
+}
+
+static void
+fcm_dcbd_timeout(void *arg)
+{
+	if (fcm_clif->cl_ping_pending > 0) {
+		fcm_dcbd_request("D");	/* DETACH_CMD */
+		fcm_dcbd_disconnect();
+	}
+	if (fcm_clif->cl_fd < 0) {
+		fcm_dcbd_connect();
+		fcm_dcbd_request("A");	/* ATTACH_CMD: attach for events */
+	} else {
+		if (!is_query_in_progress()) {
+			fcm_clif->cl_ping_pending++;
+			fcm_dcbd_request("P");	/* ping to verify connection */
+		}
+	}
+	sa_timer_set(&fcm_dcbd_timer, FCM_DCBD_TIMEOUT_USEC);
+}
+
+static void
+fcm_dcbd_disconnect(void)
+{
+	if (fcm_clif != NULL && fcm_clif->cl_local.sun_path[0] != '\0') {
+		if (fcm_clif->cl_fd >= 0)
+			sa_select_rem_fd(fcm_clif->cl_fd);
+		unlink(fcm_clif->cl_local.sun_path);
+		fcm_clif->cl_local.sun_path[0] = '\0';
+		fcm_clif->cl_fd = -1;	/* mark as disconnected */
+		fcm_clif->cl_busy = 0;
+		if (fcm_dcbd_debug)
+			SA_LOG("disconnected from dcbd");
+	}
+}
+
+static void
+fcm_dcbd_shutdown(void)
+{
+	struct fcm_fcoe *ff;
+	struct fcoe_port_config *p;
+
+	fcm_dcbd_request("D");	/* DETACH_CMD */
+	fcm_dcbd_disconnect();
+
+	p = fcoe_config.port;
+	while (p) {
+		if (p->fcoe_enable && p->dcb_required) {
+			ff = fcm_fcoe_lookup_name(p->ifname);
+			if (fcm_dcbd_debug)
+				SA_LOG("Shut down %s\n", p->ifname);
+			fcm_dcbd_setup(ff, 0);
+		}
+		p = p->next;
+	}
+
+	unlink(fcm_pidfile);
+	closelog();
+}
+
+static void
+fcm_dcbd_update(void)
+{
+	fcm_dcbd_next();
+}
+
+
+static u_int32_t
+fcm_get_hex(char *cp, u_int32_t len, char **endptr)
+{
+	u_int32_t hex = 0;
+
+	while (len > 0) {
+		len--;
+		if (*cp >= '0' && *cp <= '9')
+			hex = (hex << 4) | (*cp - '0');
+		else if (*cp >= 'A' && *cp <= 'F')
+			hex = (hex << 4) | (*cp - 'A' + 10);
+		else if (*cp >= 'a' && *cp <= 'f')
+			hex = (hex << 4) | (*cp - 'a' + 10);
+		else
+			break;
+		cp++;
+	}
+	*endptr = (len == 0) ? NULL : cp;
+	return hex;
+}
+
+static struct sa_nameval fcm_dcbd_states[] = FCM_DCBD_STATES;
+
+static void
+fcm_dcbd_state_set(struct fcm_fcoe *ff, enum fcm_dcbd_state new_state)
+{
+	if (fcm_dcbd_debug) {
+		char old[32];
+		char new[32];
+
+		SA_LOG("%s: %s -> %s",
+			ff->ff_name,
+			sa_enum_decode(old, sizeof(old),
+					fcm_dcbd_states, ff->ff_dcbd_state),
+			sa_enum_decode(new, sizeof(new),
+					fcm_dcbd_states, new_state));
+	}
+	ff->ff_dcbd_state = new_state;
+}
+
+static void
+fcm_dcbd_rx(void *arg)
+{
+	struct fcm_clif *clif = arg;
+	cmd_status st;
+	char buf[128];
+	size_t len;
+	int rc;
+	char *ep;
+
+	len = sizeof(buf);
+	rc = read(clif->cl_fd, buf, sizeof(buf) - 1);
+	if (rc < 0)
+		SA_LOG_ERR(errno, "read");
+	else if ((rc > 0) && (rc < sizeof(buf))) {
+		ASSERT(rc < sizeof(buf));
+		buf[rc] = '\0';
+		len = strlen(buf);
+		ASSERT(len <= rc);
+		if (fcm_dcbd_debug)
+			SA_LOG("received len %d buf '%s'", len, buf);
+
+		switch (buf[CLIF_RSP_MSG_OFF]) {
+		case CMD_RESPONSE:
+			st = fcm_get_hex(buf + CLIF_STAT_OFF, CLIF_STAT_LEN,
+					 &ep);
+			if (ep != NULL)
+				SA_LOG("unexpected response code from dcbd: "
+					"len %d buf %s rc %d", len, buf, rc);
+			else if (st != cmd_success &&
+				 st != cmd_device_not_found) {
+				SA_LOG("error response from dcbd: "
+					"error %d len %d %s",
+					st, len, buf);
+			}
+			fcm_clif->cl_busy = 0;
+
+			switch (buf[3]) {
+			case DCB_CMD:
+				fcm_dcbd_cmd_resp(buf, st);
+				break;
+			case ATTACH_CMD:
+				break;
+			case DETACH_CMD:
+				break;
+			case PING_CMD:
+				if (clif->cl_ping_pending > 0)
+					--clif->cl_ping_pending;
+				break;
+			case LEVEL_CMD:
+				break;
+			default:
+				SA_LOG("Unexpected cmd in response "
+					"from dcbd: len %d %s",
+					len, buf);
+				break;
+			}
+			fcm_dcbd_next();	/* advance ports if possible */
+			break;
+
+		case EVENT_MSG:
+			fcm_dcbd_event(buf, len);
+			break;
+		default:
+			SA_LOG("Unexpected message from dcbd: len %d buf %s",
+				len, buf);
+			break;
+		}
+	}
+}
+
+static void
+fcm_dcbd_ex(void *arg)
+{
+	if (fcm_dcbd_debug)
+		SA_LOG("called");
+}
+
+static void
+fcm_dcbd_request(char *req)
+{
+	size_t len;
+	int rc;
+
+	if (fcm_clif->cl_fd < 0)
+		return;
+	len = strlen(req);
+	ASSERT(fcm_clif->cl_busy == 0);
+	fcm_clif->cl_busy = 1;
+	rc = write(fcm_clif->cl_fd, req, len);
+	if (rc < 0) {
+		SA_LOG_ERR(errno, "Failed write req %s len %d", req, len);
+		fcm_clif->cl_busy = 0;
+		fcm_dcbd_disconnect();
+		fcm_dcbd_connect();
+		return;
+	}
+
+	if (fcm_dcbd_debug)
+		SA_LOG("sent '%s', rc=%d bytes succeeded", req, rc);
+	return;
+}
+
+/*
+ * Find port for message.
+ * The port name length starts at len_off for len_len bytes.
+ * The entire message length is len.
+ * The pointer to the message pointer is passed in, and updated to point
+ * past the interface name.
+ */
+static struct fcm_fcoe *
+fcm_dcbd_get_port(char **msgp, size_t len_off, size_t len_len, size_t len)
+{
+	struct fcm_fcoe *ff;
+	u_int32_t if_len;
+	char *ep;
+	char *msg;
+	char ifname[IFNAMSIZ];
+
+	msg = *msgp;
+	if (len_off + len_len >= len)
+		return NULL;
+
+	if_len = fcm_get_hex(msg + len_off, len_len, &ep);
+	if (ep != NULL) {
+		SA_LOG("Parse error on port len: msg %s", msg);
+		return NULL;
+	}
+
+	if (len_off + len_len + if_len > len) {
+		SA_LOG("Invalid port len %d msg %s", if_len, msg);
+		return NULL;
+	}
+	msg += len_off + len_len;
+	sa_strncpy_safe(ifname, sizeof(ifname), msg, if_len);
+	*msgp = msg + if_len;
+	ff = fcm_fcoe_lookup_name(ifname);
+	if (ff == NULL) {
+		SA_LOG("ifname '%s' not found", ifname);
+		exit(1);	/* XXX */
+	}
+	return ff;
+}
+
+/*
+ * (XXX) Notes:
+ * This routine is here to help fcm_dcbd_cmd_resp() to pick up
+ * information of the response packet from the DCBD. In the
+ * future, it should be merged into fcm_dcbd_cmd_resp().
+ */
+static int
+dcb_rsp_parser(struct fcm_fcoe *ff, char *rsp, cmd_status st)
+{
+	int version;
+	int dcb_cmd;
+	int feature;
+	int subtype;
+	int plen;
+	int doff;
+	int i;
+	int n;
+	struct feature_info *f_info;
+	char buf[20];
+
+	if (st != cmd_success)
+		return -1;
+
+	feature = hex2int(rsp+DCB_FEATURE_OFF);
+	if (feature != FEATURE_PFC &&
+	    feature != FEATURE_APP &&
+	    feature != FEATURE_LLINK)
+		return -1;
+
+	dcb_cmd = hex2int(rsp+DCB_CMD_OFF);
+	if (dcb_cmd != CMD_GET_CONFIG &&
+	    dcb_cmd != CMD_GET_OPER &&
+	    dcb_cmd != CMD_GET_PEER)
+		return -1;
+
+	version = rsp[DCB_VER_OFF] & 0x0f;
+	if (version != CLIF_MSG_VERSION)
+		return -1;
+
+	subtype = hex2int(rsp+DCB_SUBTYPE_OFF);
+	plen = hex2int(rsp+DCB_PORTLEN_OFF);
+	doff = DCB_PORT_OFF + plen;
+
+	switch (feature) {
+	case FEATURE_PFC:
+		f_info = &ff->ff_pfc_info;
+		break;
+	case FEATURE_APP:
+		f_info = &ff->ff_app_info;
+		f_info->subtype = subtype;
+		break;
+
+	case FEATURE_LLINK:
+		f_info = &ff->ff_llink_info;
+		f_info->subtype = subtype;
+		break;
+	}
+
+	switch (dcb_cmd) {
+	case CMD_GET_CONFIG:
+		f_info->enable = (*(rsp+doff+CFG_ENABLE) == '1');
+		f_info->advertise = (*(rsp+doff+CFG_ADVERTISE) == '1');
+		f_info->willing = (*(rsp+doff+CFG_WILLING) == '1');
+		doff += CFG_LEN;
+		break;
+
+	case CMD_GET_OPER:
+		f_info->op_vers = hex2int(rsp+doff+OPER_OPER_VER);
+		f_info->op_error = hex2int(rsp+doff+OPER_ERROR);
+		f_info->op_mode = (*(rsp+doff+OPER_OPER_MODE) == '1');
+		f_info->syncd = (*(rsp+doff+OPER_SYNCD) == '1');
+		doff += OPER_LEN;
+		if (feature == FEATURE_PFC) {
+			f_info->u.pfcup = 0;
+			for (i = 0; i < MAX_USER_PRIORITIES; i++) {
+				if (*(rsp+doff+PFC_UP(i)) == '1')
+					f_info->u.pfcup |= 1<<i;
+			}
+		}
+		if (feature == FEATURE_APP && subtype == APP_FCOE_STYPE) {
+			n = hex2int(rsp+doff+APP_LEN);
+			snprintf(buf, sizeof(buf), "%*.*s\n",
+				n, n, rsp+doff+APP_DATA);
+			f_info->u.appcfg = hex2int(buf);
+		}
+		break;
+	case CMD_GET_PEER:
+		doff += PEER_LEN;
+		if (feature == FEATURE_LLINK && subtype == LLINK_FCOE_STYPE)
+			ff->ff_llink_status = (*(rsp+doff+LLINK_STATUS) == '1');
+		break;
+	}
+
+	return 0;
+}
+
+/*
+ * validating_app_pfc - Validating App:FCoE and PFC requirements
+ *
+ * DCB is configured correctly when
+ * 1) The local configuration of the App:FCoE feature is
+ *    configured to Enable=TRUE, Advertise=TRUE, Willing=TRUE.
+ * 2) App:FCoE feature is in Opertional Mode = TRUE,
+ * 3) PFC feasture is in Opertional Mode = TRUE,
+ * 4) The priority indicated by the App:FCoE Operational Configuration
+ *    is also enabled in the PFC Operational Configuration.
+ *
+ * Returns:  1 if succeeded
+ *           0 if failed
+ */
+static int
+validating_app_pfc(struct fcm_fcoe *ff)
+{
+	if (fcm_dcbd_debug) {
+		SA_LOG("\tff_app_info.enable=%d\n",
+			ff->ff_app_info.enable);
+		SA_LOG("\tff_app_info.willing=%d\n",
+			ff->ff_app_info.op_mode);
+		SA_LOG("\tff_app_info.willing=%d\n",
+			ff->ff_app_info.op_mode);
+		SA_LOG("\tff_pfc_info.op_mode=%d\n",
+			ff->ff_pfc_info.op_mode);
+		SA_LOG("\tff_pfc_info.u.pfcup=0x%02x\n",
+			ff->ff_pfc_info.u.pfcup);
+		SA_LOG("\tff_app_info.u.appcfg=0x%02x\n",
+			ff->ff_app_info.u.appcfg);
+	}
+
+	if (!ff->ff_app_info.willing ||
+	    !ff->ff_app_info.advertise ||
+	    !ff->ff_app_info.enable)
+		return 0;
+
+	if ((!ff->ff_app_info.op_mode || \
+	     !ff->ff_pfc_info.op_mode) || \
+	     ((ff->ff_pfc_info.u.pfcup & \
+	    ff->ff_app_info.u.appcfg) != ff->ff_app_info.u.appcfg)) {
+		SA_LOG("DCB is not configured correctly\n");
+		return 0;
+	}
+
+	if (fcm_dcbd_debug)
+		SA_LOG("DCB is configured correctly\n");
+	return 1;
+}
+
+/*
+ * validating_llink_tlv - Validating Logical Link TLV requirements
+ *
+ * Logical Link TLV feature is configured correctly when
+ * 1) The local configuration of the Logical Link TLV feature is
+ *    configured to Enable=TRUE, Advertise=TRUE, Willing=TRUE.
+ * 2) The Opertional Mode of the Logical Link TLV feature must be TRUE,
+ * 3) The Link Status of the Logical Link TLV feature must be TRUE (UP).
+ *
+ * Returns:  1 if succeeded
+ *           0 if failed
+ */
+static int
+validating_llink_tlv(struct fcm_fcoe *ff)
+{
+	if (fcm_dcbd_debug) {
+		SA_LOG("\tff_llink_info.enable=%d\n",
+			ff->ff_llink_info.enable);
+		SA_LOG("\tff_llink_info.advertise=%d\n",
+			ff->ff_llink_info.advertise);
+		SA_LOG("\tff_llink_info.willing=%d\n",
+			ff->ff_llink_info.willing);
+		SA_LOG("\tff_llink_info.op_mode=%d\n",
+			ff->ff_llink_info.op_mode);
+		SA_LOG("\tff_llink_status=%d\n",
+			ff->ff_llink_status);
+	}
+
+	if ((!ff->ff_llink_info.enable  ||
+	     !ff->ff_llink_info.advertise ||
+	     !ff->ff_llink_info.willing) ||
+	     (!ff->ff_llink_info.op_mode)) {
+		SA_LOG("FCoE Logical Link is not configured correctly\n");
+		return 0;
+	}
+	if (fcm_dcbd_debug)
+		SA_LOG("FCoE Logical Link is configured correctly\n");
+
+	/*
+	 * At this point, this should be the link status
+	 * reported by the switch.
+	 */
+	if (!ff->ff_llink_status) {
+		SA_LOG("Switch Reports FCoE Logical Link is DOWN\n");
+		return 0;
+	}
+	if (fcm_dcbd_debug)
+		SA_LOG("Switch Reports FCoE Logical Link is up\n");
+
+	return 1;
+}
+
+/*
+ * validating_dcbd_info - Validating DCBD configuration and status
+ *
+ * Returns:  1 if succeeded
+ *           0 if failed
+ */
+static int
+validating_dcbd_info(struct fcm_fcoe *ff)
+{
+	int rc;
+
+	rc = validating_app_pfc(ff);
+	if (!rc)
+		return rc;
+	rc = validating_llink_tlv(ff);
+
+	return rc;
+}
+
+/*
+ * is_pfcup_changed - Check to see if PFC priority is changed
+ *
+ * Returns:  0 if no
+ *           1 if yes, but it is the first time, or was destroyed.
+ *           2 if yes
+ */
+static int
+is_pfcup_changed(struct fcm_fcoe *ff)
+{
+	if (ff->ff_pfc_info.u.pfcup != ff->ff_pfc_saved.u.pfcup) {
+		if (ff->ff_pfc_saved.u.pfcup == 0xffff)
+			return 1;	/* first time */
+		else
+			return 2;
+	}
+	return 0;
+}
+
+/*
+ * update_saved_pfcup - Update the saved PFC priority with
+ *                      the current priority.
+ *
+ * Returns:  None
+ */
+static void
+update_saved_pfcup(struct fcm_fcoe *ff)
+{
+	ff->ff_pfc_saved.u.pfcup = ff->ff_pfc_info.u.pfcup;
+}
+
+/*
+ * clear_dcbd_info - lear dcbd info to unknown values
+ *
+ */
+static void
+clear_dcbd_info(struct fcm_fcoe *ff)
+{
+	ff->ff_app_info.advertise = 0;
+	ff->ff_app_info.enable = 0;
+	ff->ff_app_info.op_mode = 0;
+	ff->ff_app_info.u.appcfg = 0;
+	ff->ff_app_info.willing = 0;
+	ff->ff_llink_info.advertise = 0;
+	ff->ff_llink_info.enable = 0;
+	ff->ff_llink_info.op_mode = 0;
+	ff->ff_llink_info.willing = 0;
+	ff->ff_llink_status = 0;
+	ff->ff_pfc_info.op_mode = 0;
+	ff->ff_pfc_info.u.pfcup = 0xffff;
+}
+
+/*
+ * Handle command response.
+ * Response buffer points past command code character in response.
+ */
+static void
+fcm_dcbd_cmd_resp(char *resp, cmd_status st)
+{
+	struct fcm_fcoe *ff;
+	u_int32_t ver;
+	u_int32_t cmd;
+	u_int32_t val;
+	u_int32_t feature;
+	u_int32_t subtype;
+	char *ep;
+	char *cp;
+	size_t len;
+	char *orig_resp;
+	u_int32_t enable;
+	u_int32_t parm_len;
+	u_int32_t parm;
+	int rc;
+
+	orig_resp = resp;
+	resp += CLIF_RSP_OFF;
+	len = strlen(resp);
+	ver = fcm_get_hex(resp + DCB_VER_OFF, DCB_VER_LEN, &ep);
+	if (ep != NULL) {
+		SA_LOG("parse error: resp %s", resp - 4);
+		return;
+	} else if (ver != CLIF_RSP_VERSION) {
+		SA_LOG("unexpected version %d resp %s", ver, orig_resp);
+		return;
+	}
+	cmd = fcm_get_hex(resp + DCB_CMD_OFF, DCB_CMD_LEN, &ep);
+	if (ep != NULL) {
+		SA_LOG("parse error on resp cmd: resp %s", orig_resp);
+		return;
+	}
+	feature = fcm_get_hex(resp + DCB_FEATURE_OFF, DCB_FEATURE_LEN, &ep);
+	if (ep != NULL) {
+		SA_LOG("parse error on resp feature: resp %s", orig_resp);
+		return;
+	}
+	subtype = fcm_get_hex(resp + DCB_SUBTYPE_OFF, DCB_SUBTYPE_LEN, &ep);
+	if (ep != NULL) {
+		SA_LOG("parse error on resp subtype: resp %s", orig_resp);
+		return;
+	}
+	cp = resp;
+	ff = fcm_dcbd_get_port(&cp, DCB_PORTLEN_OFF, DCB_PORTLEN_LEN, len);
+	if (ff == NULL) {
+		SA_LOG("port not found. resp %s", orig_resp);
+		return;
+	}
+
+	switch (cmd) {
+	case CMD_GET_CONFIG:
+		switch (ff->ff_dcbd_state) {
+		case FCD_GET_PFC_CONFIG:
+			if (st != cmd_success) {
+				fcm_dcbd_state_set(ff, FCD_ERROR);
+				break;
+			}
+			rc = dcb_rsp_parser(ff, resp, st);
+			fcm_dcbd_state_set(ff, FCD_GET_LLINK_CONFIG);
+			break;
+		case FCD_GET_LLINK_CONFIG:
+			if (st != cmd_success) {
+				fcm_dcbd_state_set(ff, FCD_ERROR);
+				break;
+			}
+			rc = dcb_rsp_parser(ff, resp, st);
+			fcm_dcbd_state_set(ff, FCD_GET_APP_CONFIG);
+			break;
+		case FCD_GET_APP_CONFIG:
+			if (st != cmd_success) {
+				fcm_dcbd_state_set(ff, FCD_ERROR);
+				break;
+			}
+			rc = dcb_rsp_parser(ff, resp, st);
+			fcm_dcbd_state_set(ff, FCD_GET_PFC_OPER);
+			break;
+		default:
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		break;
+
+	case CMD_GET_OPER:
+		/*
+		 * Sample msg: R00C103050004eth8010100100208
+		 *                  opppssll    vvmmeemsllpp
+		 * cp points past the interface name.
+		 */
+		val = fcm_get_hex(cp + OPER_ERROR, 2, &ep);
+		if (ep != NULL) {
+			SA_LOG("invalid get oper response parse error byte %d."
+				"  resp %s", ep - cp, cp);
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		if (val != 0) {
+			if (fcm_dcbd_debug) {
+				SA_LOG("resp:%s\n", orig_resp);
+				print_errors("", val);
+			}
+			/* fcm_dcbd_setup(ff, 0); */
+			fcm_dcbd_state_set(ff, FCD_DONE);
+			return;
+		}
+		if (st != cmd_success) {
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		enable = (cp[OPER_OPER_MODE] == '1');
+		switch (ff->ff_dcbd_state) {
+		case FCD_GET_PFC_OPER:
+			if (fcm_dcbd_debug) {
+				SA_LOG("%s PFC feature is %ssynced",
+				       ff->ff_name,
+				       cp[OPER_SYNCD] == '1' ? "" : "not ");
+				SA_LOG("%s PFC operating mode is %s",
+					ff->ff_name, cp[OPER_OPER_MODE] == '1'
+					? "on" : "off ");
+			}
+			ff->ff_pfc_info.enable = enable;
+			rc = dcb_rsp_parser(ff, resp, st);
+			fcm_dcbd_state_set(ff, FCD_GET_LLINK_OPER);
+			break;
+		case FCD_GET_LLINK_OPER:
+			if (fcm_dcbd_debug) {
+				SA_LOG("%s LLINK feature is %ssynced",
+				       ff->ff_name,
+				       cp[OPER_SYNCD] == '1' ? "" : "not ");
+				SA_LOG("%s LLINK operating mode is %s",
+					ff->ff_name, cp[OPER_OPER_MODE] == '1'
+					? "on" : "off ");
+			}
+			ff->ff_llink_info.enable = enable;
+			rc = dcb_rsp_parser(ff, resp, st);
+			fcm_dcbd_state_set(ff, FCD_GET_LLINK_PEER);
+			break;
+		case FCD_GET_APP_OPER:
+			if (fcm_dcbd_debug) {
+				SA_LOG("%s FCoE feature is %ssynced",
+					ff->ff_name,
+					cp[OPER_SYNCD] == '1' ? "" : "not ");
+				SA_LOG("%s FCoE operating mode is %s",
+					ff->ff_name,
+					cp[OPER_OPER_MODE] == '1' ?
+					"on" : "off ");
+			}
+			rc = dcb_rsp_parser(ff, resp, st);
+
+			parm_len = fcm_get_hex(cp + OPER_LEN, 2, &ep);
+			cp += OPER_LEN + 2;
+			if (ep != NULL || parm_len > strlen(cp)) {
+				SA_LOG("invalid peer parm_len %d", parm_len);
+				fcm_dcbd_state_set(ff, FCD_ERROR);
+				break;
+			}
+			parm = 0;
+			if (parm_len > 0) {
+				parm = fcm_get_hex(cp, parm_len, &ep);
+				if (ep != NULL) {
+					SA_LOG("invalid parameter %s", cp);
+					fcm_dcbd_state_set(ff, FCD_ERROR);
+					break;
+				}
+			}
+			ff->ff_qos_mask = parm;
+			if (validating_dcbd_info(ff)) {
+				if (fcm_dcbd_debug)
+					SA_LOG("DCB settings of %s "
+						"qualified for creating "
+						"FCoE interface\n",
+						ff->ff_name);
+				rc = is_pfcup_changed(ff);
+				if (rc == 1) {
+					if (fcm_dcbd_debug)
+						SA_LOG("%s: Initial "
+							"QOS = 0x%x\n",
+							ff->ff_name,
+							ff->ff_qos_mask);
+					fcm_dcbd_setup(ff, 1);
+				} else if (rc == 2) {
+					if (fcm_dcbd_debug)
+						SA_LOG("%s: QOS changed"
+							" to 0x%x\n",
+							ff->ff_name,
+							ff->ff_qos_mask);
+					fcm_dcbd_setup(ff, 2);
+				} else if (!ff->ff_enabled) {
+					if (fcm_dcbd_debug)
+						SA_LOG("%s: Re-create "
+							"QOS = 0x%x\n",
+							ff->ff_name,
+							ff->ff_qos_mask);
+					fcm_dcbd_setup(ff, 1);
+				} else {
+					if (fcm_dcbd_debug)
+						SA_LOG("%s: No action will "
+							"be taken\n",
+							ff->ff_name);
+				}
+			} else {
+				if (fcm_dcbd_debug)
+					SA_LOG("DCB settings of %s not "
+						"qualified for FCoE "
+						"operations.",
+						ff->ff_name);
+				fcm_dcbd_setup(ff, 0);
+				clear_dcbd_info(ff);
+			}
+
+			update_saved_pfcup(ff);
+			fcm_dcbd_state_set(ff, FCD_DONE);
+			return;
+		default:
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		break;
+
+	case CMD_GET_PEER:
+		val = fcm_get_hex(cp + OPER_ERROR, 2, &ep);
+		if (ep != NULL) {
+			SA_LOG("invalid get oper response parse error byte %d."
+				"  resp %s", ep - cp, cp);
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		if (val != 0) {
+			if (fcm_dcbd_debug) {
+				SA_LOG("resp:%s\n", orig_resp);
+				print_errors("", val);
+			}
+			/* fcm_dcbd_setup(ff, 0); */
+			fcm_dcbd_state_set(ff, FCD_DONE);
+			return;
+		}
+		if (st != cmd_success) {
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		switch (ff->ff_dcbd_state) {
+		case FCD_GET_LLINK_PEER:
+			rc = dcb_rsp_parser(ff, resp, st);
+			if (fcm_dcbd_debug) {
+				SA_LOG("%s Peer LLINK link status is %s",
+				       ff->ff_name,
+				       ff->ff_llink_status ? "up" : "down");
+			}
+			fcm_dcbd_state_set(ff, FCD_GET_APP_OPER);
+			break;
+		default:
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		break;
+
+	default:
+		SA_LOG("Unknown cmd 0x%x in response: resp %s",
+			cmd, orig_resp);
+		break;
+	}
+}
+
+static void
+fcm_event_timeout(void *arg)
+{
+	struct fcm_fcoe *ff = (struct fcm_fcoe *)arg;
+
+	if (fcm_dcbd_debug)
+		SA_LOG("%s: %d milliseconds timeout!\n",
+			ff->ff_name, FCM_EVENT_TIMEOUT_USEC/1000);
+
+	if (!is_query_in_progress()) {
+		fcm_clif->cl_ping_pending++;
+		fcm_dcbd_request("P");
+	}
+	fcm_dcbd_state_set(ff, FCD_GET_PFC_CONFIG);
+}
+
+/*
+ * Handle incoming DCB event message.
+ * Example message: E5104eth8050001
+ */
+static void
+fcm_dcbd_event(char *msg, size_t len)
+{
+	struct fcm_fcoe *ff;
+	u_int32_t feature;
+	u_int32_t subtype;
+	char *cp;
+	char *ep;
+
+	if (msg[EV_LEVEL_OFF] != MSG_DCB + '0' || len <= EV_PORT_ID_OFF)
+		return;
+	if (msg[EV_VERSION_OFF] != CLIF_EV_VERSION + '0') {
+		SA_LOG("Unexpected version in event msg %s", msg);
+		return;
+	}
+	cp = msg;
+	ff = fcm_dcbd_get_port(&cp, EV_PORT_LEN_OFF, EV_PORT_LEN_LEN, len);
+	if (ff == NULL)
+		return;
+	feature = fcm_get_hex(cp + EV_FEATURE_OFF, 2, &ep);
+	if (ep != NULL) {
+		SA_LOG("%s: Invalid feature code in event msg %s",
+			ff->ff_name, msg);
+		return;
+	}
+
+	switch (feature) {
+	case FEATURE_DCB:
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got DCB Event>\n", ff->ff_name);
+		goto ignore_event;
+	case FEATURE_PG:     /* 'E5204eth2020001' */
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got PG Event>\n", ff->ff_name);
+		goto ignore_event;
+	case FEATURE_BCN:    /* 'E5204eth2040001' */
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got BCN Event>\n", ff->ff_name);
+		goto ignore_event;
+	case FEATURE_PG_DESC:
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got PG_DESC Event>\n", ff->ff_name);
+		goto ignore_event;
+	case FEATURE_PFC:    /* 'E5204eth2030011' */
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got PFC Event>\n", ff->ff_name);
+		goto handle_event;
+	case FEATURE_APP:    /* 'E5204eth2050011' */
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got APP Event>\n", ff->ff_name);
+		goto handle_event;
+	case FEATURE_LLINK:
+		if (fcm_dcbd_debug)
+			SA_LOG("<%s: Got LLINK Event>\n", ff->ff_name);
+handle_event:
+		subtype = fcm_get_hex(cp + EV_SUBTYPE_OFF, 2, &ep);
+		if (ep != NULL || subtype != APP_FCOE_STYPE) {
+			SA_LOG("%s: Unknown application subtype in msg %s",
+				ff->ff_name, msg);
+			break;
+		}
+		if (fcm_dcbd_debug) {
+			if (cp[EV_OP_MODE_CHG_OFF] == '1')
+				SA_LOG("%s: operational mode changed",
+					ff->ff_name);
+			if (cp[EV_OP_CFG_CHG_OFF] == '1')
+				SA_LOG("%s: operational config changed",
+					ff->ff_name);
+		}
+		if (ff->ff_dcbd_state == FCD_DONE ||
+		    ff->ff_dcbd_state == FCD_ERROR) {
+			if (cp[EV_OP_MODE_CHG_OFF] == '1' ||
+			    cp[EV_OP_CFG_CHG_OFF] == '1') {
+				/* Cancel timer if it is active */
+				sa_timer_cancel(&ff->ff_event_timer);
+				/* Reset the timer */
+				sa_timer_set(&ff->ff_event_timer,
+						FCM_EVENT_TIMEOUT_USEC);
+			}
+			if (fcm_clif->cl_busy == 0)
+				fcm_dcbd_port_advance(ff);
+		}
+		break;
+	default:
+ignore_event:
+		SA_LOG("%s: Unknown feature 0x%x in msg %s",
+			ff->ff_name, feature, msg);
+		break;
+	}
+}
+
+/*
+ * Run script to enable or disable the interface or print a message.
+ *
+ * Input:  enable = 0      Destroy the FCoE interface
+ *         enable = 1      Create the FCoE interface
+ *         enable = 2      Reset the interface
+ */
+static void
+fcm_dcbd_setup(struct fcm_fcoe *ff, u_int32_t enable)
+{
+	char *op;
+	char *qos_arg;
+	char qos[64];
+	u_int32_t mask;
+	int rc;
+	int fd;
+
+	if (enable == 0)
+		op = "--disable";
+	else if (enable == 1)
+		op = "--enable";
+	else
+		op = "--reset";
+	if (enable && !ff->ff_qos_mask)
+		return;
+	if (fcm_dcbd_cmd == NULL) {
+		SA_LOG("Should %s %s per op state", op, ff->ff_name);
+		return;
+	}
+	/*
+	 * XXX should wait for child status
+	 */
+	ff->ff_enabled = enable;
+
+	rc = fork();
+	if (rc < 0) {
+		SA_LOG_ERR(errno, "fork error");
+	} else if (rc == 0) {	/* child process */
+		for (fd = ulimit(4 /* __UL_GETOPENMAX */ , 0); fd > 2; fd--)
+			close(fd);
+		qos_arg = NULL;
+		if (enable) {
+			mask = ff->ff_qos_mask;
+			if (mask) {
+				int off = 0;
+				char *sep = "";
+				u_int32_t bit;
+
+				while (mask != 0 && off < sizeof(qos) - 1) {
+					bit = ffs(mask) - 1;
+					off +=
+					    snprintf(qos + off,
+						     sizeof(qos) - off, "%s%u",
+						     sep, bit);
+					mask &= ~(1 << bit);
+					sep = ",";
+				}
+				qos_arg = "--qos";
+			}
+		}
+		if (fcm_dcbd_debug) {
+			if (!enable)
+				SA_LOG("%s %s %s\n",
+					fcm_dcbd_cmd, ff->ff_name, op);
+			else
+				SA_LOG("%s %s %s %s %s\n",
+					fcm_dcbd_cmd, ff->ff_name, op,
+					qos_arg, qos);
+		}
+		execlp(fcm_dcbd_cmd, fcm_dcbd_cmd, ff->ff_name,
+		       op, qos_arg, qos, (char *)NULL);
+		SA_LOG_ERR(errno, "exec '%s' failed", fcm_dcbd_cmd);
+		exit(1);
+	}
+}
+
+/*
+ * Called for all ports.  For FCoE ports and candidates,
+ * get information and send to dcbd.
+ */
+static void
+fcm_dcbd_port_advance(struct fcm_fcoe *ff)
+{
+	char buf[80];
+	struct fcoe_port_config *p;
+
+	ASSERT(ff);
+	ASSERT(fcm_clif);
+
+	p = fcm_find_port_config(ff->ff_name);
+	if (!p)
+		return;
+
+	if (ff->ff_dcbd_state != FCD_INIT && !fcm_fcoe_port_ready(ff))
+		fcm_dcbd_state_set(ff, FCD_INIT);
+	if (fcm_clif->cl_busy)
+		return;
+
+	switch (ff->ff_dcbd_state) {
+	case FCD_INIT:
+		if (!fcm_fcoe_port_ready(ff)) {
+			if (fcm_dcbd_debug)
+				SA_LOG("FCoE port %s not ready\n", ff->ff_name);
+			fcm_dcbd_state_set(ff, FCD_ERROR);
+			break;
+		}
+		fcm_fcoe_get_dcb_settings(ff);
+		fcm_dcbd_state_set(ff, FCD_GET_PFC_CONFIG);
+		break;
+	case FCD_GET_PFC_CONFIG:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_CONFIG, FEATURE_PFC, 0,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_LLINK_CONFIG:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_CONFIG, FEATURE_LLINK, LLINK_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_APP_CONFIG:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_CONFIG, FEATURE_APP, APP_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_PFC_OPER:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_OPER, FEATURE_PFC, 0,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_LLINK_OPER:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_OPER, FEATURE_LLINK, LLINK_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_LLINK_PEER:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_PEER, FEATURE_LLINK, LLINK_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_APP_OPER:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_OPER, FEATURE_APP, APP_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_GET_PEER:
+		snprintf(buf, sizeof(buf), "%c%x%2.2x%2.2x%2.2x%2.2x%s%s",
+			DCB_CMD, CLIF_RSP_VERSION,
+			CMD_GET_PEER, FEATURE_APP, APP_FCOE_STYPE,
+			(u_int) strlen(ff->ff_name), ff->ff_name, "");
+		fcm_dcbd_request(buf);
+		break;
+	case FCD_DONE:
+		break;
+	case FCD_ERROR:
+		break;
+	default:
+		break;
+	}
+}
+
+static void
+fcm_dcbd_next(void)
+{
+	struct fcm_fcoe *ff;
+
+	TAILQ_FOREACH(ff, &fcm_fcoe_head, ff_list) {
+		if (fcm_clif->cl_busy)
+			break;
+		fcm_dcbd_port_advance(ff);
+	}
+}
+
+static void
+fcm_usage(void)
+{
+	printf("%s\n", fcoemon_version);
+	printf("Usage: %s\n"
+		"\t [-e|--exec <exec>]\n"
+		"\t [-d|--debug]\n"
+		"\t [-v|--version]\n"
+		"\t [-h|--help]\n\n", progname);
+	exit(1);
+}
+
+static void
+fcm_sig(int sig)
+{
+	fcm_dcbd_shutdown();
+	sa_select_exit();
+}
+
+static void
+fcm_pidfile_create(void)
+{
+	FILE *fp;
+	char buf[100];
+	char *sp;
+	int pid;
+	int rc;
+
+	fp = fopen(fcm_pidfile, "r+");
+	if (fp) {
+		sp = fgets(buf, sizeof(buf), fp);
+		pid = atoi(sp);
+		rc = kill(pid, 0);
+		if (sp && (pid > 0) && !rc) {
+			SA_LOG("Another instance"
+				" (pid %d) is running - exiting\n",
+				pid);
+			exit(1);
+		}
+		fclose(fp);
+	}
+	fp = fopen(fcm_pidfile, "w+");
+	if (fp) {
+		fprintf(fp, "%d\n", getpid());
+		fclose(fp);
+	}
+}
+
+int main(int argc, char **argv)
+{
+	struct sigaction sig;
+	int rc;
+	int c;
+
+	strncpy(progname, basename(argv[0]), sizeof(progname));
+	sa_log_prefix = progname;
+	sa_log_flags = 0;
+	openlog(sa_log_prefix, LOG_CONS, LOG_DAEMON);
+
+	while ((c = getopt_long(argc, argv, "de:hv",
+			fcm_options, NULL)) != -1) {
+		switch (c) {
+		case 'd':
+			fcm_dcbd_debug = 1;
+			break;
+		case 'e':
+			fcm_dcbd_cmd = optarg;
+			break;
+		case 'v':
+			printf("%s\n", fcoemon_version);
+			return 0;
+		case 'h':
+		default:
+			fcm_usage();
+			break;
+		}
+	}
+	if (argc != optind)
+		fcm_usage();
+
+	/*
+	 * Set up for signals.
+	 */
+	memset(&sig, 0, sizeof(sig));
+	sig.sa_handler = fcm_sig;
+	rc = sigaction(SIGINT, &sig, NULL);
+	if (rc < 0)
+		SA_LOG_ERR_EXIT(errno, "sigaction failed");
+	rc = sigaction(SIGTERM, &sig, NULL);
+	if (rc < 0)
+		SA_LOG_ERR_EXIT(errno, "sigaction failed");
+	rc = sigaction(SIGHUP, &sig, NULL);
+	if (rc < 0)
+		SA_LOG_ERR_EXIT(errno, "sigaction failed");
+	fcm_pidfile_create();
+	fcm_fcoe_init();
+	fcm_link_init();	/* NETLINK_ROUTE protocol */
+	fcm_dcbd_init();
+
+	sa_select_loop();
+	fcm_dcbd_shutdown();
+	return 0;
+}
+
+/*******************************************************
+ *         The following are debug routines            *
+ *******************************************************/
+
+static void
+print_errors(char *buf, int errors)
+{
+	char msg[80];
+	int len, j;
+	int flag = 0;
+
+	memset(msg, 0, sizeof(msg));
+	len = sprintf(msg, "0x%02x - ", errors);
+
+	if (!errors) {
+		j = sprintf(msg + len, "none\n");
+		SA_LOG("%s %s", buf, msg);
+		return;
+	}
+
+	if (errors & 0x01) {
+		flag++;
+		j = sprintf(msg + len, "mismatch with peer");
+	}
+
+	if (errors & 0x02) {
+		j = len;
+		if (flag++)
+			j = sprintf(msg + len, ", ");
+		sprintf(msg + j, "local configuration error");
+	}
+
+	if (errors & 0x04) {
+		j = len;
+		if (flag++)
+			j = sprintf(msg + len, ", ");
+		sprintf(msg + j, "multiple TLV's received");
+	}
+
+	if (errors & 0x08) {
+		j = len;
+		if (flag++)
+			j = sprintf(msg + len, ", ");
+		sprintf(msg + j, "peer error");
+	}
+
+	if (errors & 0x10) {
+		j = len;
+		if (flag++)
+			j = sprintf(msg + len, ", ");
+		sprintf(msg + j, "multiple LLDP neighbors");
+	}
+
+	if (errors & 0x20) {
+		j = len;
+		if (flag++)
+			j = sprintf(msg + len, ", ");
+		sprintf(msg + j, "peer feature not present");
+	}
+
+	SA_LOG_ERR(errors, "%s %s\n", buf, msg);
+}
+
diff --git a/usr/tools/fcoemon/fcoemon.h b/usr/tools/fcoemon/fcoemon.h
new file mode 100644
index 0000000..43ce67d
--- /dev/null
+++ b/usr/tools/fcoemon/fcoemon.h
@@ -0,0 +1,115 @@
+/*
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Maintained at www.Open-FCoE.org
+ */
+
+#ifndef _FCOEMON_H_
+#define _FCOEMON_H_
+
+/*
+ * States for HBAs relative to the DCB daemon.
+ * States advance sequentially if conditions are right.
+ */
+enum fcm_dcbd_state {
+   FCD_INIT = 0,        /* starting state */
+   FCD_GET_PFC_CONFIG,  /* getting PFC configuration */
+   FCD_GET_LLINK_CONFIG,/* getting LLINK configuration */
+   FCD_GET_APP_CONFIG,  /* getting APP configuration */
+   FCD_GET_PFC_OPER,    /* getting PFC operational mode */
+   FCD_GET_LLINK_OPER,  /* getting LLINK operational mode */
+   FCD_GET_LLINK_PEER,  /* getting LLINK peer configuration */
+   FCD_GET_APP_OPER,    /* getting operational mode */
+   FCD_GET_PEER,        /* getting peer configuration */
+   FCD_DONE,            /* DCB exchanges complete */
+   FCD_ERROR,           /* DCB error or port unknown by DCB */
+};
+
+#define FCM_DCBD_STATES {                         \
+    { "INIT",             FCD_INIT },             \
+    { "GET_PFC_CONFIG",   FCD_GET_PFC_CONFIG },   \
+    { "GET_LLINK_CONFIG", FCD_GET_LLINK_CONFIG }, \
+    { "GET_APP_CONFIG",   FCD_GET_APP_CONFIG },   \
+    { "GET_PFC_OPER",     FCD_GET_PFC_OPER },     \
+    { "GET_LLINK_OPER",   FCD_GET_LLINK_OPER },   \
+    { "GET_LLINK_PEER",   FCD_GET_LLINK_PEER },   \
+    { "GET_APP_OPER",     FCD_GET_APP_OPER },     \
+    { "GET_PEER",         FCD_GET_PEER },   \
+    { "DONE",             FCD_DONE },             \
+    { "ERROR",            FCD_ERROR },            \
+    { NULL,               0 }                     \
+}
+
+struct feature_info {
+   u_int32_t	enable;    /* enable/disable feature */
+   u_int32_t	advertise; /* enable/disable advertise */
+   u_int32_t	willing;   /* enable/disable willing mode */
+   u_int32_t	syncd;     /* synchronized with switch */
+   u_int32_t	op_mode;   /* operational mode */
+   u_int32_t	op_vers;   /* feature operational version */
+   u_int32_t	op_error;  /* operational error */
+   u_int32_t	subtype;   /* subtype */
+   union {
+      u_int32_t pfcup;
+      u_int32_t appcfg;
+   } u;
+};
+
+/*
+ * Description of potential FCoE interface.
+ */
+struct fcm_fcoe {
+   TAILQ_ENTRY(fcm_fcoe) ff_list;          /* list linkage */
+   u_int32_t             ff_ifindex;       /* kernel interface index */
+   u_int32_t             ff_flags;         /* kernel interface flags */
+   u_int32_t             ff_last_flags;    /* previously known flags */
+   u_int32_t             ff_enabled:1;     /* operational status */
+   struct feature_info   ff_pfc_info;      /* PFC feature info */
+   struct feature_info   ff_pfc_saved;     /* saved PFC feature info */
+   struct feature_info   ff_app_info;      /* App feature info */
+   struct feature_info   ff_llink_info;    /* LLink feature info */
+   u_int32_t             ff_llink_status;  /* LLink status */
+   u_int32_t             ff_has_fip;       /* FIP is implemented */
+   u_int64_t             ff_mac;           /* MAC address */
+   int                   ff_vlan;          /* VLAN ID or -1 if none */
+   u_int8_t              ff_operstate;     /* RFC 2863 operational status */
+   u_int8_t              ff_qos_mask;      /* 801.p priority mask */
+   enum fcm_dcbd_state   ff_dcbd_state;    /* DCB daemon state */
+   struct sa_timer       ff_event_timer;   /* Event timer */
+   char                  ff_name[IFNAMSIZ];/* Ethernet interface name */
+};
+
+TAILQ_HEAD(fcm_fcoe_head, fcm_fcoe);
+
+struct fcm_fcoe_head fcm_fcoe_head;
+extern char build_date[];
+
+static void fcm_dcbd_init(void);
+static void fcm_dcbd_shutdown(void);
+static void fcm_dcbd_update(void);
+static void fcm_fcoe_init(void);
+#ifdef NOT_YET
+static struct fcm_fcoe *fcm_fcoe_lookup_mac(u_int64_t ff_mac, int vlan);
+static struct fcm_fcoe *fcm_fcoe_lookup_create_mac(u_int64_t ff_mac, int vlan);
+#endif
+static struct fcm_fcoe *fcm_fcoe_lookup_name(char *name);
+static struct fcm_fcoe *fcm_fcoe_lookup_create_ifindex(u_int32_t ifindex);
+static void fcm_fcoe_set_name(struct fcm_fcoe *, char *);
+static void fcm_fcoe_get_dcb_settings(struct fcm_fcoe *);
+static int fcm_fcoe_port_ready(struct fcm_fcoe *);
+static int fcm_link_init(void);
+
+#endif /* _FCOEMON_H_ */
diff --git a/usr/tools/fcoemon/fcoemon_utils.c b/usr/tools/fcoemon/fcoemon_utils.c
new file mode 100644
index 0000000..e98d259
--- /dev/null
+++ b/usr/tools/fcoemon/fcoemon_utils.c
@@ -0,0 +1,760 @@
+/*
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Maintained at www.Open-FCoE.org
+ */
+
+#include "fcoemon_utils.h"
+#include "net_types.h"
+#include "fc_types.h"
+
+u_char libsa_lock_hier;		/* for lock debugging non-log related */
+
+/*
+ * Size of on-stack line buffers.
+ * These shouldn't be to large for a kernel stack frame.
+ */
+#define SA_LOG_BUF_LEN  200	/* on-stack line buffer size */
+
+/*
+ * log with a variable argument list.
+ */
+static void
+sa_log_va(const char *func, const char *format, va_list arg)
+{
+	size_t len;
+	size_t flen;
+	int add_newline;
+	char sa_buf[SA_LOG_BUF_LEN];
+	char *bp;
+
+	/*
+	 * If the caller didn't provide a newline at the end, we will.
+	 */
+	len = strlen(format);
+	add_newline = 0;
+	if (!len || format[len - 1] != '\n')
+		add_newline = 1;
+	bp = sa_buf;
+	len = sizeof(sa_buf);
+	if (func) {
+		flen = snprintf(bp, len, "%s: ", func);
+		len -= flen;
+		bp += flen;
+	}
+	flen = vsnprintf(bp, len, format, arg);
+	if (add_newline && flen < len) {
+		bp += flen;
+		*bp++ = '\n';
+		*bp = '\0';
+	}
+	sa_log_output(sa_buf);
+}
+
+/*
+ * log
+ */
+void
+sa_log(const char *format, ...)
+{
+	va_list arg;
+
+	va_start(arg, format);
+	sa_log_va(NULL, format, arg);
+	va_end(arg);
+}
+
+/*
+ * log with function name.
+ */
+void
+sa_log_func(const char *func, const char *format, ...)
+{
+	va_list arg;
+
+	va_start(arg, format);
+	sa_log_va(func, format, arg);
+	va_end(arg);
+}
+
+/*
+ * log with error number.
+ */
+void
+sa_log_err(int error, const char *func, const char *format, ...)
+{
+	va_list arg;
+	char buf[SA_LOG_BUF_LEN];
+
+	if (func)
+		sa_log("%s: error %d %s", func, error,
+		       strerror_r(error, buf, sizeof(buf)));
+	else
+		sa_log("error %d %s", error,
+		       strerror_r(error, buf, sizeof(buf)));
+	va_start(arg, format);
+	sa_log_va(func, format, arg);
+	va_end(arg);
+}
+
+/*
+ * Size of on-stack line buffers.
+ * These shouldn't be to large for a kernel stack frame.
+ */
+#define SA_LOG_BUF_LEN  200	/* on-stack line buffer size */
+
+/*
+ * Assert failures.
+ */
+void
+assert_failed(const char *format, ...)
+{
+	va_list arg;
+	char buf[SA_LOG_BUF_LEN];
+
+	va_start(arg, format);
+	vsnprintf(buf, sizeof(buf), format, arg);
+	va_end(arg);
+	sa_log_abort(buf);
+}
+
+/*
+ * Log options.
+ * These may be set directly by callers.
+ */
+u_int sa_log_flags;                     /* timestamp and other option flags */
+int sa_log_time_delta_min = 1;          /* minimum diff to print in millisec */
+char *sa_log_prefix;                    /* string to print before any message */
+
+void
+sa_log_set_option(u_int flags)
+{
+	sa_log_flags = flags;
+}
+
+/*
+ * Put timestamp on front of each log line, as controlled by tunables above.
+ */
+static void
+sa_log_timestamp(void)
+{
+	static struct timeval tlast;
+	char ctime_buf[30];
+	struct timeval t;
+	struct timeval diff;
+
+	gettimeofday(&t, NULL);
+	if (sa_log_flags & SA_LOGF_TIME) {
+		ctime_r(&t.tv_sec, ctime_buf);
+		ctime_buf[11 + 8] = '\0';   /* trim ctime after seconds */
+		fprintf(stderr, "%s.%3.3ld ",
+			ctime_buf + 11, t.tv_usec / 1000);
+	}
+	if (sa_log_flags & SA_LOGF_DELTA) {
+		if (tlast.tv_sec == 0)
+			tlast = t;
+		timersub(&t, &tlast, &diff);
+		tlast = t;
+		if (diff.tv_sec != 0 ||
+		    diff.tv_usec >= sa_log_time_delta_min * 1000)
+			fprintf(stderr, "%4ld.%3.3ld ",
+				diff.tv_sec, diff.tv_usec / 1000);
+		else
+			fprintf(stderr, "%8s ", "");
+	}
+	if (sa_log_prefix)
+		fprintf(stderr, "%s: ", sa_log_prefix);
+}
+
+void
+sa_log_output(const char *buf)
+{
+	if (fcm_use_syslog) {
+		syslog(LOG_INFO, "%s", buf);
+		return;
+	}
+	sa_log_timestamp();
+	fprintf(stderr, "%s", buf);
+	fflush(stderr);
+}
+
+void
+sa_log_abort(const char *buf)
+{
+	sa_log_output(buf);
+	abort();
+}
+
+void
+sa_log_output_exit(const char *buf)
+{
+	exit(1);
+}
+
+/*
+ * Make a printable NUL-terminated copy of the string.
+ * The source buffer might not be NUL-terminated.
+ */
+char *
+sa_strncpy_safe(char *dest, size_t len, const char *src, size_t src_len)
+{
+	char *dp = dest;
+	const char *sp = src;
+
+	while (len-- > 1 && src_len-- > 0 && *sp != '\0') {
+		*dp++ = isprint(*sp) ? *sp : (isspace(*sp) ? ' ' : '.');
+		sp++;
+	}
+	*dp = '\0';
+
+	/*
+	 * Take off trailing blanks.
+	 */
+	while (--dp >= dest && isspace(*dp))
+		*dp = '\0';
+	return dest;
+}
+
+/** sa_enum_decode(buf, len, tp, val)
+ *
+ * @param buf buffer for result (may be used or not).
+ * @param len size of buffer (at least 32 bytes recommended).
+ * @param tp pointer to table of names and values, struct sa_nameval.
+ * @param val value to be decoded into a name.
+ * @returns pointer to name string.  Unknown values are put into buffer in hex.
+ */
+const char *
+sa_enum_decode(char *buf, size_t len, const struct sa_nameval *tp, u_int val)
+{
+	for (; tp->nv_name != NULL; tp++) {
+		if (tp->nv_val == val)
+			return tp->nv_name;
+	}
+	snprintf(buf, len, "Unknown (code 0x%X)", val);
+	return buf;
+}
+
+#define SA_TIMER_HZ     (1000 * 1000 * 1000ULL)	/* nanoseconds per second */
+#define SA_TIMER_FUZZ   (500 * 1000ULL)	/* 500 microseconds is close enough */
+
+static struct sa_timer *sa_timer_head;	/* queue of scheduled events */
+static u_int64_t sa_timer_nsec;		/* nanoseconds since start */
+
+/*
+ * Initialize a timer structure.  Set handler.
+ */
+void
+sa_timer_init(struct sa_timer *tm, void (*handler)(void *), void *arg)
+{
+	ASSERT(handler != NULL);
+	memset(tm, 0, sizeof(*tm));
+	tm->tm_handler = handler;
+	tm->tm_handler_arg = arg;
+}
+
+/*
+ * Allocate a timer structure.  Set handler.
+ */
+struct sa_timer *
+sa_timer_alloc(void (*handler)(void *arg), void *arg)
+{
+	struct sa_timer *tm;
+
+	tm = malloc(sizeof(*tm));
+	if (tm)
+		sa_timer_init(tm, handler, arg);
+	return tm;
+}
+
+u_int64_t
+sa_timer_get(void)
+{
+	u_int64_t nsec;
+#ifndef _POSIX_TIMERS
+	struct timeval tv;
+
+	gettimeofday(&tv, NULL);	/* XXX want monotonic time, not TOD */
+	nsec = tv.tv_sec * SA_TIMER_HZ + tv.tv_usec * 1000;
+#else /* _POSIX_TIMERS */
+	struct timespec ts;
+	int rc;
+
+	rc = clock_gettime(CLOCK_MONOTONIC, &ts);
+	ASSERT_NOTIMPL(rc == 0);
+	nsec = ts.tv_sec * SA_TIMER_HZ + ts.tv_nsec;
+#endif /* _POSIX_TIMERS */
+
+#if 0 /* XXX */
+	ASSERT(nsec >= sa_timer_nsec);	/* really must be monotonic */
+#else
+	if (nsec < sa_timer_nsec)
+		sa_log("sa_timer_get: negative time lapse "
+			"old %qud new %qud diff %qd nsec\n",
+			(long long unsigned int) sa_timer_nsec,
+			(long long unsigned int) nsec,
+			(long long int) (nsec - sa_timer_nsec));
+#endif
+	sa_timer_nsec = nsec;
+	return nsec;
+}
+
+/*
+ * Get monotonic time since some arbitrary time in the past.
+ * If _POSIX_MONOTONIC_CLOCK isn't available, we'll use time of day.
+ */
+u_int
+sa_timer_get_secs(void)
+{
+	u_int sec;
+
+#ifndef _POSIX_TIMERS
+	struct timeval tv;
+
+	gettimeofday(&tv, NULL); /* XXX want monotonic time, not TOD */
+	sec = tv.tv_sec;
+#else /* _POSIX_TIMERS */
+	struct timespec ts;
+	int rc;
+
+	rc = clock_gettime(CLOCK_MONOTONIC, &ts);
+	ASSERT_NOTIMPL(rc == 0);
+	sec = ts.tv_sec;
+#endif /* _POSIX_TIMERS */
+	return sec;
+}
+
+/*
+ * Set timer to fire.   Delta is in microseconds from now.
+ */
+void
+sa_timer_set(struct sa_timer *tm, u_long delta_usec)
+{
+	struct sa_timer *cur;
+	struct sa_timer **prev;
+
+	ASSERT(delta_usec != 0);
+	ASSERT(tm->tm_handler != NULL);
+	sa_timer_cancel(tm);
+	ASSERT(sa_timer_active(tm) == 0);
+	tm->tm_nsec =
+	    sa_timer_get() + delta_usec * SA_TIMER_HZ / SA_TIMER_UNITS;
+	ASSERT(tm->tm_nsec != 0);
+
+	/*
+	 * Insert timer into sorted linked list.
+	 * Find insertion point, before cur.
+	 */
+	for (prev = &sa_timer_head;
+	     (cur = *prev) != NULL && cur->tm_nsec <= tm->tm_nsec;
+	     prev = &cur->tm_next)
+		;
+	*prev = tm;
+	tm->tm_next = cur;
+}
+
+/*
+ * Cancel timer if it is active.
+ */
+void
+sa_timer_cancel(struct sa_timer *tm)
+{
+	struct sa_timer *cur;
+	struct sa_timer **prev;
+
+	if (sa_timer_active(tm)) {
+		for (prev = &sa_timer_head; (cur = *prev) != NULL;
+		     prev = &cur->tm_next)
+			if (cur == tm) {
+				tm->tm_nsec = 0;
+				*prev = tm->tm_next;
+				break;
+			}
+		ASSERT(cur == tm);
+	}
+}
+
+/*
+ * Free (and cancel) timer.
+ */
+void
+sa_timer_free(struct sa_timer *tm)
+{
+	if (sa_timer_active(tm))
+		sa_timer_cancel(tm);
+	free(tm);
+}
+
+/*
+ * Handle timer checks.  Called from select loop or other periodic function.
+ *
+ * The struct timeval is set before returning to the maximum amount of time
+ * that should elapse before the next call.
+ *
+ * Returns 1 if any timer functions were called, 0 otherwise.
+ */
+int
+sa_timer_check(struct timeval *tv)
+{
+	u_int64_t now = 0;
+	u_int64_t next_due = 0;
+	struct sa_timer *tm;
+	int ret = 0;
+
+	/*
+	 * Remember, the list may change during the handler.
+	 */
+	for (;;) {
+		now = sa_timer_get();
+		tm = sa_timer_head;
+		if (tm == NULL) {
+			next_due = now;
+			break;
+		}
+
+		next_due = tm->tm_nsec;
+		if (next_due > now + SA_TIMER_FUZZ)
+			break;
+
+		/*
+		 * Remove this element from the list.
+		 */
+		sa_timer_head = tm->tm_next;
+		tm->tm_next = NULL;
+
+		/*
+		 * Mark cancelled and call handler.
+		 */
+		tm->tm_nsec = 0;
+		ASSERT(tm->tm_handler != NULL);
+		(*tm->tm_handler)(tm->tm_handler_arg);
+		ret = 1;
+	}
+
+	ASSERT(next_due >= now);
+	next_due -= now;
+	tv->tv_sec = (time_t) (next_due / SA_TIMER_HZ);
+	tv->tv_usec = (long) (next_due % SA_TIMER_HZ) / 1000;
+
+	return ret;
+}
+
+#define NFC_NFDS        64
+
+/*
+ * Deferred procedure call.
+ */
+struct sa_defer_ent {
+	TAILQ_ENTRY(sa_defer_ent) de_next;
+	void        (*de_func)(void *arg);
+	void        *de_arg;
+};
+
+/*
+	* Static module state.
+	*/
+static struct sa_sel_state {
+	fd_set      ts_rx_fds;
+	fd_set      ts_tx_fds;
+	fd_set      ts_ex_fds;
+	int         ts_max_fd;
+	u_char      ts_exit;
+	struct sa_sel_fd {
+		void    (*ts_rx_handler)(void *);
+		void    (*ts_tx_handler)(void *);
+		void    (*ts_ex_handler)(void *);
+		void    *ts_handler_arg;
+	} ts_fd[NFC_NFDS];
+	void        (*ts_callback)(void);
+	TAILQ_HEAD(, sa_defer_ent) ts_defer_list;
+} sa_sel_state;
+
+static void
+sa_select_call_deferred_funcs(void)
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+	struct sa_defer_ent *de, *de_next;
+
+	de = ss->ts_defer_list.tqh_first;
+	TAILQ_INIT(&ss->ts_defer_list);
+
+	for (; de != NULL; de = de_next) {
+		de_next = de->de_next.tqe_next;
+		if (de->de_func != NULL)
+			(*de->de_func)(de->de_arg);
+		free(de);
+	}
+}
+
+void
+sa_select_loop(void)
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+	struct sa_sel_fd *fp;
+	fd_set rx_fds;
+	fd_set tx_fds;
+	fd_set ex_fds;
+	struct timeval tval;
+	struct timeval *tvp;
+	int rv, i;
+
+	ss->ts_exit = 0;
+	while (ss->ts_exit == 0) {
+		sa_timer_check(&tval);
+		if (ss->ts_exit)
+			break;
+		if (ss->ts_defer_list.tqh_first != NULL) {
+			/*
+			 * If a timer or deferred function added a new deferred
+			 * function, just poll through select (zero-timeout).
+			 */
+			tval.tv_sec = tval.tv_usec = 0;
+			tvp = &tval;
+		} else if (tval.tv_sec == 0 && tval.tv_usec == 0)
+			tvp = NULL;
+		else
+			tvp = &tval;
+		rx_fds = ss->ts_rx_fds;
+		tx_fds = ss->ts_tx_fds;
+		ex_fds = ss->ts_ex_fds;
+		rv = select(ss->ts_max_fd + 1, &rx_fds, &tx_fds, &ex_fds, tvp);
+		if (rv == -1) {
+			if (errno == EINTR)
+				continue;
+			SA_LOG_ERR_EXIT(errno, "select error");
+		}
+
+		fp = ss->ts_fd;
+		for (i = 0; rv > 0 && i <= sa_sel_state.ts_max_fd; i++, fp++) {
+			if (FD_ISSET(i, &rx_fds)) {
+				if (fp->ts_rx_handler != NULL)
+					(*fp->ts_rx_handler)
+					(fp->ts_handler_arg);
+				else
+					ASSERT(!FD_ISSET(i, &ss->ts_rx_fds));
+				--rv;
+			}
+			if (FD_ISSET(i, &tx_fds)) {
+				if (fp->ts_tx_handler != NULL)
+					(*fp->ts_tx_handler)
+					(fp->ts_handler_arg);
+				else
+					ASSERT(!FD_ISSET(i, &ss->ts_tx_fds));
+				--rv;
+			}
+			if (FD_ISSET(i, &ex_fds)) {
+				if (fp->ts_ex_handler != NULL)
+					(*fp->ts_ex_handler)
+					(fp->ts_handler_arg);
+				else
+					ASSERT(!FD_ISSET(i, &ss->ts_ex_fds));
+				--rv;
+			}
+		}
+		if (ss->ts_callback != NULL)
+			(*ss->ts_callback)();
+		if (ss->ts_defer_list.tqh_first != NULL)
+			sa_select_call_deferred_funcs();
+	}
+}
+
+void
+sa_select_add_fd(int fd,
+		 void (*rx_handler)(void *),
+		 void (*tx_handler)(void *),
+		 void (*ex_handler)(void *),
+		 void *arg)
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+	struct sa_sel_fd *fp;
+
+	ASSERT_NOTIMPL(fd < NFC_NFDS);
+	ASSERT(rx_handler != NULL || tx_handler != NULL || ex_handler != NULL);
+	if (ss->ts_max_fd < fd)
+		ss->ts_max_fd = fd;
+	fp = &ss->ts_fd[fd];
+	fp->ts_handler_arg = arg;
+	if (rx_handler != NULL) {
+		fp->ts_rx_handler = rx_handler;
+		FD_SET(fd, &ss->ts_rx_fds);
+	}
+	if (tx_handler != NULL) {
+		fp->ts_tx_handler = tx_handler;
+		FD_SET(fd, &ss->ts_tx_fds);
+	}
+	if (ex_handler != NULL) {
+		fp->ts_ex_handler = ex_handler;
+		FD_SET(fd, &ss->ts_ex_fds);
+	}
+}
+
+void
+sa_select_set_rx(int fd, void (*handler)(void *))
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+
+	ASSERT(fd <= ss->ts_max_fd);
+	ss->ts_fd[fd].ts_rx_handler = handler;
+	if (handler != NULL)
+		FD_SET(fd, &ss->ts_rx_fds);
+	else
+		FD_CLR(fd, &ss->ts_rx_fds);
+}
+
+void
+sa_select_set_tx(int fd, void (*handler)(void *))
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+
+	ASSERT(fd <= ss->ts_max_fd);
+	ss->ts_fd[fd].ts_tx_handler = handler;
+	if (handler != NULL)
+		FD_SET(fd, &ss->ts_tx_fds);
+	else
+		FD_CLR(fd, &ss->ts_tx_fds);
+}
+
+void
+sa_select_set_ex(int fd, void (*handler)(void *))
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+
+	ASSERT(fd <= ss->ts_max_fd);
+	ss->ts_fd[fd].ts_ex_handler = handler;
+	if (handler != NULL)
+		FD_SET(fd, &ss->ts_ex_fds);
+	else
+		FD_CLR(fd, &ss->ts_ex_fds);
+}
+
+void
+sa_select_rem_fd(int fd)
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+	struct sa_sel_fd *fp;
+
+	ASSERT_NOTIMPL(fd < NFC_NFDS);
+	FD_CLR(fd, &ss->ts_rx_fds);
+	FD_CLR(fd, &ss->ts_tx_fds);
+	FD_CLR(fd, &ss->ts_ex_fds);
+	fp = &ss->ts_fd[fd];
+	fp->ts_rx_handler = NULL;
+	fp->ts_tx_handler = NULL;
+	fp->ts_ex_handler = NULL;
+	fp->ts_handler_arg = NULL;
+}
+
+/*
+ * Set callback for every time through the select loop.
+ */
+void
+sa_select_set_callback(void (*cb)(void))
+{
+	sa_sel_state.ts_callback = cb;
+}
+
+/*
+ * Add a deferred function call.
+ */
+void *
+sa_select_add_deferred_callback(void (*func)(void *), void *arg)
+{
+	struct sa_sel_state *ss = &sa_sel_state;
+	struct sa_defer_ent *de;
+
+	ASSERT(func != NULL);
+
+	de = malloc(sizeof(*de));
+	if (de != NULL) {
+		de->de_func = func;
+		de->de_arg = arg;
+		if (ss->ts_defer_list.tqh_first == NULL)
+			TAILQ_INIT(&ss->ts_defer_list);
+		TAILQ_INSERT_TAIL(&ss->ts_defer_list, de, de_next);
+	}
+
+	return de;
+}
+
+/*
+ * Delete (cancel) a deferred function call.
+ */
+void
+sa_select_del_deferred_callback(void *handle)
+{
+	struct sa_defer_ent *de = handle;
+
+	de->de_func = NULL;
+}
+
+/*
+ * Cause select loop to exit.
+ * This is invoked from a handler which wants the select loop to return
+ * after the handler is finished.  For example, during receipt of a network
+ * packet, the program may decide to clean up and exit, but in order to do
+ * this cleanly, all lower-level protocol handlers should return first.
+ */
+void
+sa_select_exit(void)
+{
+	sa_sel_state.ts_exit = 1;
+}
+
+/*
+ * Convert 48-bit IEEE MAC address to 64-bit FC WWN.
+ */
+fc_wwn_t
+fc_wwn_from_mac(u_int64_t mac, u_int scheme, u_int port)
+{
+	fc_wwn_t wwn;
+
+	ASSERT(mac < (1ULL << 48));
+	wwn = mac | ((fc_wwn_t) scheme << 60);
+	switch (scheme) {
+	case 1:
+		ASSERT(port == 0);
+		break;
+	case 2:
+		ASSERT(port < 0xfff);
+		wwn |= (fc_wwn_t) port << 48;
+		break;
+	default:
+		ASSERT_NOTREACHED;
+		break;
+	}
+	return wwn;
+}
+
+/* assumes input is pointer to two hex digits */
+/* returns -1 on error */
+int
+hex2int(char *b)
+{
+	int i;
+	int n = 0;
+	int m;
+
+	for (i = 0, m = 1; i < 2; i++, m--) {
+		if (isxdigit(*(b+i))) {
+			if (*(b+i) <= '9')
+				n |= (*(b+i) & 0x0f) << (4*m);
+			else
+				n |= ((*(b+i) & 0x0f) + 9) << (4*m);
+		} else
+			return -1;
+	}
+	return n;
+}
+
diff --git a/usr/tools/fcoemon/fcoemon_utils.h b/usr/tools/fcoemon/fcoemon_utils.h
new file mode 100644
index 0000000..e8c5de9
--- /dev/null
+++ b/usr/tools/fcoemon/fcoemon_utils.h
@@ -0,0 +1,315 @@
+/*
+ * Copyright(c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Maintained at www.Open-FCoE.org
+ */
+
+#ifndef _FCOEMON_UTILS_H_
+#define _FCOEMON_UTILS_H_
+
+#include <sys/types.h>
+#include <sys/time.h>
+#include <sys/socket.h>
+#include <sys/queue.h>
+#include <malloc.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <stdarg.h>
+#include <ctype.h>
+#include <time.h>
+#include <unistd.h>
+#include <signal.h>
+#include <errno.h>
+#include <syslog.h>
+
+#include "fc_types.h"
+
+void sa_log(const char *format, ...);
+void sa_log_func(const char *func, const char *format, ...);
+void sa_log_err(int, const char *func, const char *format, ...);
+
+/*
+ * These functions can be provided outside of libsa for those environments
+ * that want to redirect them.
+ */
+void sa_log_output(const char *);	/* log message */
+void sa_log_abort(const char *);	/* log message and abort */
+void sa_log_output_exit(const char *);	/* log message and exit */
+
+#define __SA_STRING(x)  #x
+
+/*
+ * Log message.
+ */
+#define SA_LOG(...) \
+	do {								\
+		sa_log_func(__func__, __VA_ARGS__);			\
+	} while (0)
+
+#define SA_LOG_ERR(error, ...) \
+	do {								\
+		sa_log_err(error, NULL, __VA_ARGS__);			\
+	} while (0)
+
+/*
+ * Logging exits.
+ */
+#define SA_LOG_EXIT(...) \
+	do {								\
+		sa_log_func(__func__, __VA_ARGS__);			\
+		sa_log_func(__func__, "exiting at %s:%d",		\
+			__FILE__, __LINE__);				\
+		sa_log_output_exit(__func__);				\
+	} while (0)
+
+#define SA_LOG_ERR_EXIT(error, ...) \
+	do {								\
+		sa_log_func(__func__, __VA_ARGS__);			\
+		sa_log_err(error, __func__, "exiting at %s:%d",		\
+			__FILE__, __LINE__);				\
+		sa_log_output_exit(__func__);           		\
+	} while (0)
+
+/*
+ * Logging options.
+ */
+#define SA_LOGF_TIME    0x0001      /* include timestamp in message */
+#define SA_LOGF_DELTA   0x0002      /* include time since last message */
+
+extern u_int sa_log_flags;          /* timestamp and other option flags */
+extern int sa_log_time_delta_min;   /* minimum diff to print in millisec */
+extern char *sa_log_prefix;         /* string to print before any message */
+
+extern void assert_failed(const char *s, ...)
+    __attribute__ ((format(printf, 1, 2)));
+
+#ifndef UNLIKELY
+#define UNLIKELY(_x) (_x)
+#endif /* UNLIKELY */
+
+/*
+ * ASSERT macros
+ *
+ * ASSERT(expr) - this calls assert_failed() if expr is false.  This variant
+ * is not present in production code or if DEBUG_ASSERTS is not defined.
+ * Be careful not to rely on expr being evaluated.
+ */
+#if defined(DEBUG_ASSERTS)
+#define ASSERT(_x) do {							\
+		if (UNLIKELY(!(_x))) {					\
+			assert_failed("ASSERT FAILED (%s) @ %s:%d\n",	\
+				"" #_x, __FILE__, __LINE__);		\
+		}							\
+	} while (0)
+#else
+#define ASSERT(_x)
+#endif /* DEBUG_ASSERTS */
+
+/*
+ * ASSERT_NOTIMPL(expr) - this calls assert_failed() if expr is false.
+ * The implication is that the condition is not handled by the current
+ * implementation, and work should be done eventually to handle this.
+ */
+#define ASSERT_NOTIMPL(_x) do {						\
+		if (UNLIKELY(!(_x))) {					\
+			assert_failed("ASSERT (NOT IMPL) "		\
+				"(%s) @ %s:%d\n",			\
+				"" #_x, __FILE__, __LINE__);		\
+		}							\
+	} while (0)
+
+/*
+ * ASSERT_NOTREACHED - this is the same as ASSERT_NOTIMPL(0).
+ */
+#define ASSERT_NOTREACHED do {						\
+		assert_failed("ASSERT (NOT REACHED) @ %s:%d\n",		\
+			__FILE__, __LINE__);				\
+	} while (0)
+
+/*
+ * ASSERT_BUG(bugno, expr).  This variant is used when a bug number has
+ * been assigned to any one of the other assertion failures.  It is always
+ * present in code.  It gives the bug number which helps locate
+ * documentation and helps prevent duplicate bug filings.
+ */
+#define ASSERT_BUG(_bugNr, _x) do {					\
+		if (UNLIKELY(!(_x))) {					\
+			assert_failed("ASSERT (BUG %d) (%s) @ %s:%d\n", \
+				(_bugNr), #_x, __FILE__, __LINE__);	\
+		}                                                       \
+	} while (0)
+
+#ifndef LIBSA_USE_DANGEROUS_ROUTINES
+#define strcpy DONT_USE_strcpy
+#define strcat DONT_USE_strcat
+#define gets   DONT_USE_gets
+#endif /* LIBSA_USE_DANGEROUS_ROUTINES */
+
+char *sa_strncpy_safe(char *dest, size_t len, const char *src, size_t src_len);
+char *sa_hex_format(char *buf, size_t buflen,
+			const unsigned char *data, size_t data_len,
+			unsigned int group_len, char *inter_group_sep);
+
+/*
+ * Structure for tables encoding and decoding name-value pairs such as enums.
+ */
+struct sa_nameval {
+    char    *nv_name;
+    u_int   nv_val;
+};
+
+const char *sa_enum_decode(char *, size_t, const struct sa_nameval *, u_int);
+int sa_enum_encode(const struct sa_nameval *tp, const char *, u_int *);
+const char *sa_flags_decode(char *, size_t, const struct sa_nameval *, u_int);
+
+/*
+ * Timer facility.
+ */
+
+struct sa_timer {
+	struct sa_timer	*tm_next;
+	u_int64_t	tm_nsec;	/* relative time to event (nSec) */
+	void		(*tm_handler)(void *arg);
+	void		*tm_handler_arg;
+	struct sa_timer **timer_head;
+};
+
+
+#define SA_TIMER_UNITS  (1000 * 1000UL)	/* number of timer ticks per second */
+
+/*
+ * Initialize a pre-allocated timer structure.
+ */
+void sa_timer_init(struct sa_timer *, void (*handler)(void *), void *arg);
+
+/*
+ * Test whether the timer is active.
+ */
+static inline int sa_timer_active(struct sa_timer *tm)
+{
+	return tm->tm_nsec != 0;
+}
+
+/*
+ * Allocate a timer structure.  Set handler.
+ */
+struct sa_timer *sa_timer_alloc(void (*)(void *arg), void *arg);
+
+/*
+ * Set timer to fire.   Delta is in microseconds from now.
+ */
+void sa_timer_set(struct sa_timer *, u_long delta);
+
+/*
+ * Cancel timer.
+ */
+void sa_timer_cancel(struct sa_timer *);
+
+/*
+ * Free (and cancel) timer.
+ */
+void sa_timer_free(struct sa_timer *);
+
+
+/*
+ * Handle timer checks.  Called from select loop or other periodic function.
+ *
+ * The struct timeval passed in indicates how much time has passed since
+ * the last call, and is set before returning to the maximum amount of time
+ * that should elapse before the next call.
+ *
+ * Returns 1 if any timer handlers were invoked, 0 otherwise.
+ */
+int sa_timer_check(struct timeval *);
+
+/*
+ * Get time in nanoseconds since some arbitrary time.
+ */
+u_int64_t sa_timer_get(void);
+
+/*
+ * Get time in seconds since some arbitrary time.
+ */
+u_int sa_timer_get_secs(void);
+
+/*
+ * sa_select - Server Array select facility.
+ *
+ * This is a thin layer to poll files with a select loop.
+ */
+
+/*
+ * Enter the polling loop which never exits.
+ */
+void sa_select_loop(void);
+
+/*
+ * Set callback for every time through the select loop.
+ */
+void sa_select_set_callback(void (*)(void));
+
+/*
+ * Add a deferred function call.  The function is called at the start
+ * of the next select loop cycle.
+ * Returns a handle to the deferred call object on success, or NULL on memory
+ * allocation failure.
+ */
+void *sa_select_add_deferred_callback(void (*func)(void *), void *arg);
+
+/*
+ * Delete a deferred function call.
+ * Takes the handle returned by sa_select_add_deferred_callback as an argument.
+ */
+void sa_select_del_deferred_callback(void *handle);
+
+
+/*
+ * Add a callback to handle files which are ready for receive, transmit,
+ * or to handle exceptions.
+ */
+void sa_select_add_fd(int fd, void (*rx_handler)(void *),
+			void (*tx_handler)(void *),
+			void (*ex_handler)(void *), void *arg);
+
+/*
+ * Change a single callback for a descriptor that's already been added.
+ */
+void sa_select_set_rx(int fd, void (*handler)(void *));
+void sa_select_set_tx(int fd, void (*handler)(void *));
+void sa_select_set_ex(int fd, void (*handler)(void *));
+
+/*
+ * Remove all callbacks for a file descriptor.
+ */
+void sa_select_rem_fd(int fd);
+
+/*
+ * Cause select loop to return.
+ */
+void sa_select_exit(void);
+
+/*
+ * Convert 48-bit IEEE MAC address to 64-bit FC WWN.
+ */
+extern fc_wwn_t
+fc_wwn_from_mac(u_int64_t, u_int32_t scheme, u_int32_t port);
+
+extern int hex2int(char *b);
+extern int fcm_use_syslog;
+
+#endif /* _FCOEMON_UTILS_H_ */
diff --git a/usr/tools/fcoemon/fcoeplumb.sh b/usr/tools/fcoemon/fcoeplumb.sh
new file mode 100755
index 0000000..8ea7ad1
--- /dev/null
+++ b/usr/tools/fcoemon/fcoeplumb.sh
@@ -0,0 +1,345 @@
+#! /bin/bash
+#
+# Copyright(c) 2009 Intel Corporation. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# Maintained at www.Open-FCoE.org
+
+cmdname=`basename $0`
+
+usage()
+{
+	echo usage: $cmdname \
+		'<ethX> [--reset | --enable | --disable]' \
+		'[--qos <pri>[,<pri>]...]]' >&2
+	exit 1
+}
+
+#
+# Tunable parameters
+#
+QOS_DEF=3			# default user priority
+FCOE_ETHERTYPE=35078		# Ethertype (0x8906): tc filter show is base 10
+FCOE_FILTER=0xfc0e		# filter handle (must be lower-case hex)
+qdisc_id=1:
+qos_list=
+FILTER_ID=
+cmd=
+#
+TC_CMD="/usr/sbin/tc"		# /sbin/tc or debug command
+FCOEADM=/sbin/fcoeadm		# command to create/destroy FCoE instances
+LOGGER="logger -s -t fcoeplumb"
+CONFIG_DIR=/etc/fcoe
+
+. $CONFIG_DIR/config
+if [ "$USE_SYSLOG" != "yes" ] && [ "$USE_SYSLOG" != "YES" ]; then
+	LOGGER="echo"
+else
+	USE_SYSLOG="yes"
+fi
+[ "$DEBUG" = "yes" ] || [ "$DEBUG" = "YES" ] && DEBUG="yes"
+
+find_multiq_qdisc()
+{
+	ifname=$1
+
+	found=0
+	type=unknown
+	if set -- `$TC_CMD qdisc show dev $ifname` none none none
+	then
+		type=$2
+		qdisc_id=$3
+	fi
+	[ "$type" == "multiq" ] && found=1
+
+	return $found
+}
+
+add_multiq_qdisc()
+{
+	ifname=$1
+	qdisc_id=$2
+
+	[ "$DEBUG" = "yes" ] && $LOGGER \
+		"$TC_CMD qdisc add dev $ifname root handle $qdisc_id multiq"
+	$TC_CMD qdisc add dev $ifname root handle $qdisc_id multiq
+}
+
+delete_qdisc()
+{
+	ifname=$1
+
+	[ "$DEBUG" = "yes" ] && $LOGGER \
+		"$TC_CMD qdisc del dev $ifname root"
+	$TC_CMD qdisc del dev $ifname root
+}
+
+get_filter_id()
+{
+	ifname=$1
+
+	retry_count=0
+	while true
+	do
+		[ $retry_count -eq 0 ] && break
+		[ -f /var/run/fcoemon.pid ] && break
+		sleep 1
+		retry_count=$(($retry_count-1))
+	done
+
+	FILTER_ID=`echo "$ifname 12345" | \
+		awk '{ printf("0x%x%06x", substr($1,4), $2) }'`
+}
+
+find_skbedit_filter()
+{
+	ifname=$1
+
+	found=`$TC_CMD filter show dev $ifname | awk '
+	BEGIN {
+		x1 = 0
+		x2 = 0
+		x3 = 0
+		queue = 8
+	}
+	/^filter.*parent.*protocol 802_3.* handle '$FILTER_ID'/ {
+		if (x1 == 0 && x2 == 0 && x3 == 0)
+			x1 = 1
+	}
+	/cmp.*u16 at 12 layer 1 mask 0xffff eq '$FCOE_ETHERTYPE'.*\)/ {
+		if (x1 == 1 && x2 == 0 && x3 == 0)
+			x2 = 1
+	}
+	/action order [0-9][0-9]*:  skbedit queue_mapping/ {
+		if (x1 == 1 && x2 == 1 && x3 == 0) {
+			x3 = 1
+			queue = $6
+		}
+	}
+	END {
+		print queue
+	}'`
+
+	return $found
+}
+
+delete_skbedit_filter()
+{
+	ifname=$1
+	queue=$?
+
+	[ "$DEBUG" = "yes" ] && $LOGGER \
+		"$TC_CMD filter delete dev $ifname skbedit queue_mapping $queue"
+	PARENT=`$TC_CMD filter show dev $ifname | awk \
+		'/^filter.*parent.*protocol 802_3.* handle '$FILTER_ID'/ \
+		{ print $3 }'`
+	PRIO=`$TC_CMD filter show dev $ifname | awk \
+		'/^filter.*parent.*protocol 802_3.* handle '$FILTER_ID'/ \
+		{ print $7 }'`
+	$TC_CMD filter delete dev $ifname parent $PARENT \
+		protocol 802_3 pref $PRIO handle $FILTER_ID basic match \
+		'cmp(u16' at 12 layer 1 mask 0xffff eq $FCOE_ETHERTYPE')' \
+		action skbedit queue_mapping $queue
+	$TC_CMD filter delete dev $ifname parent $PARENT \
+		protocol 802_3 pref $PRIO basic
+}
+
+add_skbedit_filter()
+{
+	ifname=$1
+	qdisc_id=$2
+	queue=$3
+
+	[ "$DEBUG" = "yes" ] && $LOGGER \
+		"$TC_CMD filter add dev $ifname skbedit queue_mapping $queue"
+	$TC_CMD filter add dev $ifname parent $qdisc_id protocol 802_3 \
+		handle $FILTER_ID basic match 'cmp(u16' at 12 \
+		layer 1 mask 0xffff eq $FCOE_ETHERTYPE')' \
+		action skbedit queue_mapping $queue
+}
+
+replace_skbedit_filter()
+{
+	ifname=$1
+	queue=$2
+
+	[ "$DEBUG" = "yes" ] && $LOGGER \
+		"$TC_CMD filter replace dev $ifname skbedit queue_mapping $queue"
+	PARENT=`$TC_CMD filter show dev $ifname | awk \
+		'/^filter.*parent.*protocol 802_3.* handle '$FILTER_ID'/ \
+		{ print $3 }'`
+	PRIO=`$TC_CMD filter show dev $ifname | \
+		awk '/^filter.*parent.*protocol 802_3.* handle '$FILTER_ID'/ \
+		{ print $7 }'`
+	$TC_CMD filter replace dev $ifname parent $PARENT protocol \
+		802_3 pref $PRIO handle $FILTER_ID basic match \
+		'cmp(u16' at 12 layer 1 mask 0xffff eq $FCOE_ETHERTYPE')' \
+		action skbedit queue_mapping $queue
+}
+
+remove_fcoe_interface()
+{
+	ifname=$1
+
+	STATUS=`$FCOEADM -i $ifname 2>&1 | \
+		awk '/Interface Name:/{print $3}'`
+	if [ "$STATUS" = "$ifname" ]; then
+		[ "$DEBUG" = "yes" ] && $LOGGER "$FCOEADM -d $ifname"
+		$FCOEADM -d $ifname
+	else
+		[ "$DEBUG" = "yes" ] && $LOGGER \
+			"FCoE interface $ifname doesn't exist"
+	fi
+}
+
+create_fcoe_interface()
+{
+	ifname=$1
+
+	STATUS=`$FCOEADM -i $ifname 2>&1 | \
+		awk '/Interface Name:/{print $3}'`
+	if [ -z "$STATUS" ]; then
+		[ "$DEBUG" = "yes" ] && $LOGGER "$FCOEADM -c $ifname"
+		$FCOEADM -c $ifname
+	else
+		[ "$DEBUG" = "yes" ] && $LOGGER \
+			"FCoE interface $ifname already created"
+	fi
+}
+
+[ "$#" -lt 1 ] && usage
+
+[ "$DEBUG" = "yes" ] && $LOGGER "fcoeplumb arguments: ($*)"
+
+ifname=$1
+shift
+
+while [ "$#" -ge 1 ]
+do
+	case "$1" in
+	--reset | -r)
+		cmd=reset
+		;;
+	--enable | -e)
+		cmd=enable
+		;;
+	--disable | -d)
+		cmd=disable
+		;;
+	--debug)
+		LOGGER="logger -t fcoeplumb -s"
+		;;
+	--qos | -q)
+		[ "$#" -lt 2 ] && usage
+		qos_list=$2
+		shift
+		;;
+	*)
+		echo "$cmdname: unknown parameter '$1'" >&2
+		usage
+		;;
+	esac
+	shift
+done
+
+# This must be the first to do after parsing the command arguments!
+# Notice that the FILTER_ID is used in find_skbedit_filter(),
+# add_skbedit_filter(), replace_skbedit_filter().
+get_filter_id $ifname
+
+if [ "$cmd" == "disable" ]; then
+	remove_fcoe_interface $ifname
+	find_skbedit_filter $ifname
+	found_filter=$?
+	[ $found_filter -le 7 ] && delete_skbedit_filter $ifname $found_filter
+else
+	#
+	# Choose the best QOS to use for FCoE out of the listed choices.
+	#
+
+	# Parse QOS List
+	QOS_BEST=
+	if [ -n "$qos_list" ]; then
+		OLD_IFS="$IFS"
+		IFS=,"$IFS"
+		set -- $qos_list
+		IFS="$OLD_IFS"
+
+		while [ "$#" -ge 1 ]
+		do
+			case "$1" in
+			[0-7])
+				;;
+			*)
+				echo "$cmdname: bad QOS value '$1'" >&2
+				usage
+				;;
+			esac
+			if [ -z "$QOS_BEST" ]; then
+				QOS_BEST=$1
+			elif [ "$1" -eq "$QOS_DEF" ]; then
+				QOS_BEST=$1
+			fi
+			shift
+		done
+	fi
+
+	[ "$DEBUG" = "yes" ] && $LOGGER "$ifname - Choosing QOS '$QOS_BEST'"
+
+	# If the best QOS is not found, do nothing.
+	[ -z "$QOS_BEST" ] && exit 0
+
+	#
+	# Setup the traffic classifier for FCoE
+	# First see if it is already set up.
+	#
+	qos_queue=`expr $QOS_BEST`
+
+	find_multiq_qdisc $ifname
+	found_qdisc=$?
+
+	if [ $found_qdisc -eq 1 ]; then
+		[ "$DEBUG" = "yes" ] && $LOGGER "$ifname: Qdisc is found"
+		find_skbedit_filter $ifname
+		found_filter=$?
+		if [ $found_filter -gt 7 ]; then
+			[ "$DEBUG" = "yes" ] && $LOGGER \
+				"$ifname: Filter is not found"
+			add_skbedit_filter $ifname $qdisc_id $qos_queue
+		elif [ $found_filter -ne $qos_queue ]; then
+			[ "$DEBUG" = "yes" ] && $LOGGER \
+				"$ifname: Filter is found and QOS is different"
+			replace_skbedit_filter $ifname $qos_queue
+		else
+			[ "$DEBUG" = "yes" ] && $LOGGER \
+				"$ifname: Filter is found and is identical"
+		fi
+	else
+		[ "$DEBUG" = "yes" ] && $LOGGER "$ifname: Qdisc is not found"
+		add_multiq_qdisc $ifname $qdisc_id
+		add_skbedit_filter $ifname $qdisc_id $qos_queue
+		delete_qdisc $ifname
+		add_multiq_qdisc $ifname $qdisc_id
+		add_skbedit_filter $ifname $qdisc_id $qos_queue
+	fi
+
+	if [ "$cmd" = "enable" ]; then
+		create_fcoe_interface $ifname
+	fi
+fi
+
+[ "$DEBUG" = "yes" ] && $LOGGER "$ifname: Leaving"
+exit 0
+




More information about the devel mailing list