[Open-FCoE] [PATCH 00/11] fcoemon restructuring

Eric Multanen eric.w.multanen at intel.com
Sat Dec 5 00:51:54 UTC 2009

This patch set provides a fairly significant redesign of fcoemon.  fcoemon
was suffering from a number of issues, including being too reactive to
events.  This resulted in unstable behavior.  This note describes the high
level functions of the redesigned fcoemon.  The following patches provide the

fcoemon will now have the following high level structure:
1.  Read in FCoE interface configuration files.  This information is used
    to create a list of FCoE interfaces.  As link and dcbd events
    occur and dcbd queries are resolved, the FCoE interface list will manage
    the creation and destruction of FCoE interfaces.  Each FCoE interface
    element in the list keeps track of its network interface (from the
    config file and could be a VLAN), its real network interface (found
    from link events during runtime), and its next action (create, destroy,

2.  As link events are received from the system, fcoemon determines which ones
    are relevent for the FCoE interfaces.  Relevent link events are those which
    occur on interfaces which are configured to be FCoE interfaces (see
    the FCoE interface list above), or on the underlying real network interface,
    if the FCoE interface is configured on a VLAN.  A list of relevant real
    network interfaces is maintained.  The list is used to track the
    operstate of the real network interfaces and, if DCB is required,
    the status of dcbd queries for the real network interface.

3.  The core loop of fcoemon:
	- listens for link events from rtnetlink and updates the state of the
	  real network interface list. Link down events will terminate any
	  pending dcbd query sequences and a subsequent link up will start the
	  sequence over from the beginning.  Delete link events (VLAN deleted,
	  driver unloaded) will mark the FCoE interface for destruction.
	- listens for dcbd events and response messages.  If the response
	  matches the current dcbd query state, then move to the next state.
	  If a dcbd event indicates a change in a DCB feature of interest,
	  then start the dcbd query sequence over.
	- manages a timeout for maintaining a connection to dcbd.  If the dcbd
	  service stops, this timeout will clean up as needed and re-establish
	  the dcbd connection once dcbd starts up again.
	- at the end of each event/timeout cycle, after all link and dcbd
	  and network interface events have been handled, perform any
	  pending next actions.
	  For network interface elements, that could mean sending the next
	  dcbd query, or, if the dcbd queries are complete, analyze the
	  results and determine if the FCoE interface should be
	  will now handle all FCoE interfaces whether or not DCB is required.
	  For FCoE interfaces, this means creating or destroying the FCoE

4.  All FCoE interfaces, whether DCB is required or not, will be managed by
    fcoemon now.  Previously, the init.d start script would handle configured
    FCoE interfaces which were configured without DCB required.

dcbd query sequence
When DCB is required for an FCoE interface, the real network device performs a
series of dcbd queries to obtain the current DCB configuration.  The sequence
is as follows:
1.  DCB state - is DCB enabled on the interface?
2.  PFC configuration (Priority Flow Control)
3.  FCoE configuration
4.  PFC operational state
5.  FCoE operational state

If an error occurs at any step, or if all the steps are completed and the
dcbd state does not match the conditions to either create or destroy the
FCoE interface (see next sections), then a retry timer is started.  Once the
timer expires, the dcbd query sequence is reinitiated from the beginning.  The
timer begins at 0.2 seconds and increases by multiples of that amount up to
ten retries.  Once the max retries has been reached, without a successful
completion of the dcbd query sequence, then the FCoE interface is marked for

The function of this retry mechanism is to protect against destroying the
FCoE interface due to intermittent problems.  Such as when links go down
momentarily due to configuration changes.  When this occurs, dcbd may take
a couple seconds to resynchronize with the peer device.

Required conditions to create an FCoE interface
1.  Link is up on the network interfaces required for the FCoE interface, and

2a. DCB is not required.
2b. DCB is required
	AND DCB is enabled
	AND PFC is enabled
	AND App:FCoE is enabled
	AND PFC and App:FCoE are operational
	AND PFC and App:FCoE operational values match

Required conditions to destroy an FCoE interface
1.  The network interface required for the FCoE interface is removed
	Such as driver is unloaded or VLAN interface is destroyed.
	This is detected by DELLINK rtnetlink events.

2.  The network interfaces required for the FCoE interface are up,
	AND DCB is required,
	AND on root network interface
		DCB is disabled
		App:FCoE is disabled
		PFC and App:FCoE are operational
		AND PFC and App:FCoE operational values mismatch

3.  When DCB is required and the conditions to create the FCoE interface are
    not satisfied after going through the maximum number of retry sequences,
    as described in the "dcbd query sequence" section, then the FCoE interface
    will be destroyed.

Eric Multanen (11):
      Update version string of fcoemon.
      Remove FCoE interface management from service start script
      Add dcbd query timeout and retry mechanism
      Update the fcoemon state machine to be insensitive to intermittent dcb changes
      Update the FCoE and network interface lists.
      Add separate arguments to specify FCoE and network interface
      Fix the print_errors() routine
      Remove extraneous dcb members from the network interface structure.
      Fixup dcbd connection timeout code
      Remove unused dcbd routine.
      Obtain the real device of a VLAN interface using an ioctl.

 etc/initd/initd.fedora |   82 ---
 etc/initd/initd.suse   |   53 --
 fcoemon.c              | 1504 ++++++++++++++++++++++--------------------------
 fcoemon.h              |   67 +-
 fcoeplumb.in           |  121 ++--
 5 files changed, 789 insertions(+), 1038 deletions(-)


More information about the devel mailing list