[Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring

Shyam_Iyer at Dell.com Shyam_Iyer at Dell.com
Tue Dec 15 17:35:29 UTC 2009

> -----Original Message-----
> From: devel-bounces at open-fcoe.org [mailto:devel-bounces at open-fcoe.org]
> On Behalf Of Eric Multanen
> Sent: Tuesday, December 15, 2009 10:31 AM
> To: devel at open-fcoe.org
> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring
> v2 - restore code in the service script which destroys FCoE interfaces
> for
> which DCB is not required when the service is stopped.
> This patch set provides a fairly significant redesign of fcoemon.
> fcoemon
> was suffering from a number of issues, including being too reactive to
> events.  This resulted in unstable behavior.  This note describes the
> high
> level functions of the redesigned fcoemon.  The following patches
> provide the
> implementation.
> fcoemon will now have the following high level structure:
> 1.  Read in FCoE interface configuration files.  This information is
> used
>     to create a list of FCoE interfaces.  As link and dcbd events
>     occur and dcbd queries are resolved, the FCoE interface list will
> manage
>     the creation and destruction of FCoE interfaces.  Each FCoE
> interface
>     element in the list keeps track of its network interface (from the
>     config file and could be a VLAN), its real network interface
>     from link events during runtime), and its next action (create,
> destroy,
>     etc.).
> 2.  As link events are received from the system, fcoemon determines
> which ones
>     are relevent for the FCoE interfaces.  Relevent link events are
> those which
>     occur on interfaces which are configured to be FCoE interfaces
>     the FCoE interface list above), or on the underlying real network
> interface,
>     if the FCoE interface is configured on a VLAN.  A list of relevant
> real
>     network interfaces is maintained.  The list is used to track the
>     operstate of the real network interfaces and, if DCB is required,
>     the status of dcbd queries for the real network interface.
> 3.  The core loop of fcoemon:
> 	- listens for link events from rtnetlink and updates the state
> the
> 	  real network interface list. Link down events will terminate
> any
> 	  pending dcbd query sequences and a subsequent link up will
> start the
> 	  sequence over from the beginning.  Delete link events (VLAN
> deleted,
> 	  driver unloaded) will mark the FCoE interface for destruction.
> 	- listens for dcbd events and response messages.  If the
> 	  matches the current dcbd query state, then move to the next
> state.
> 	  If a dcbd event indicates a change in a DCB feature of
> 	  then start the dcbd query sequence over.
> 	- manages a timeout for maintaining a connection to dcbd.  If
> dcbd
> 	  service stops, this timeout will clean up as needed and re-
> establish
> 	  the dcbd connection once dcbd starts up again.
> 	- at the end of each event/timeout cycle, after all link and
> 	  and network interface events have been handled, perform any
> 	  pending next actions.
> 	  For network interface elements, that could mean sending the
> next
> 	  dcbd query, or, if the dcbd queries are complete, analyze the
> 	  results and determine if the FCoE interface should be
> 	  will now handle all FCoE interfaces whether or not DCB is
> required.
> 	  For FCoE interfaces, this means creating or destroying the
> 	  interface.
> 4.  All FCoE interfaces, whether DCB is required or not, will be
> managed by
>     fcoemon now.  Previously, the init.d start script would handle
> configured
>     FCoE interfaces which were configured without DCB required.
> dcbd query sequence
> ===================
> When DCB is required for an FCoE interface, the real network device
> performs a
> series of dcbd queries to obtain the current DCB configuration.  The
> sequence
> is as follows:
> 1.  DCB state - is DCB enabled on the interface?
> 2.  PFC configuration (Priority Flow Control)
> 3.  FCoE configuration
> 4.  PFC operational state
> 5.  FCoE operational state
> If an error occurs at any step, or if all the steps are completed and
> the
> dcbd state does not match the conditions to either create or destroy
> the
> FCoE interface (see next sections), then a retry timer is started.
> Once the
> timer expires, the dcbd query sequence is reinitiated from the
> beginning.  The
> timer begins at 0.2 seconds and increases by multiples of that amount
> up to
> ten retries.  Once the max retries has been reached, without a
> successful
> completion of the dcbd query sequence, then the FCoE interface is
> marked for
> destruction.
> The function of this retry mechanism is to protect against destroying
> the
> FCoE interface due to intermittent problems.  Such as when links go
> down
> momentarily due to configuration changes.  When this occurs, dcbd may
> take
> a couple seconds to resynchronize with the peer device.
> Required conditions to create an FCoE interface
> ===============================================
> 1.  Link is up on the network interfaces required for the FCoE
> interface, and
> 2a. DCB is not required.
> 	OR
> 2b. DCB is required
> 	AND DCB is enabled
> 	AND PFC is enabled
> 	AND App:FCoE is enabled
> 	AND PFC and App:FCoE are operational
> 	AND PFC and App:FCoE operational values match
> Required conditions to destroy an FCoE interface
> ================================================
> 1.  The network interface required for the FCoE interface is removed
> 	Such as driver is unloaded or VLAN interface is destroyed.
> 	This is detected by DELLINK rtnetlink events.
> 2.  The network interfaces required for the FCoE interface are up,
> 	AND DCB is required,
> 	AND on root network interface
> 		DCB is disabled
> 			OR
> 		App:FCoE is disabled
> 			OR
> 		PFC and App:FCoE are operational
> 		AND PFC and App:FCoE operational values mismatch
> 3.  When DCB is required and the conditions to create the FCoE
> interface are
>     not satisfied after going through the maximum number of retry
> sequences,
>     as described in the "dcbd query sequence" section, then the FCoE
> interface
>     will be destroyed.
> ---
> Eric Multanen (11):
>       Update version string of fcoemon.
>       Remove FCoE interface management from service start script
>       Add dcbd query timeout and retry mechanism
>       Update the fcoemon state machine to be insensitive to
> intermittent dcb changes
>       Update the FCoE and network interface lists.
>       Add separate arguments to specify FCoE and network interface
>       Fix the print_errors() routine
>       Remove extraneous dcb members from the network interface
> structure.
>       Fixup dcbd connection timeout code
>       Remove unused dcbd routine.
>       Obtain the real device of a VLAN interface using an ioctl.
>  etc/initd/initd.fedora |   66 --
>  etc/initd/initd.suse   |   37 -
>  fcoemon.c              | 1504
> --------
>  fcoemon.h              |   67 +-
>  fcoeplumb.in           |  121 ++--
>  5 files changed, 789 insertions(+), 1006 deletions(-)
> --
> Signature: Eric Multanen <eric.w.multanen at intel.com>
> _______________________________________________
> devel mailing list
> devel at open-fcoe.org
> http://www.open-fcoe.org/mailman/listinfo/devel

Shouldn't the requirement to administer DCB  be FCoE independent ? After
all DCB is required for other protocols like iscsi as well.

How about using a library interface to manage DCB over a network
interface so that both open-iscsi and open-fcoe could administer DCB? 

More information about the devel mailing list