[Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring

Shyam_Iyer at Dell.com Shyam_Iyer at Dell.com
Tue Dec 15 18:20:36 UTC 2009


> -----Original Message-----
> From: Multanen, Eric W [mailto:eric.w.multanen at intel.com]
> Sent: Tuesday, December 15, 2009 11:42 PM
> To: Iyer, Shyam; devel at open-fcoe.org
> Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> restructuring
> 
> >-----Original Message-----
> >From: Shyam_Iyer at Dell.com [mailto:Shyam_Iyer at Dell.com]
> >Sent: Tuesday, December 15, 2009 9:35 AM
> >To: Multanen, Eric W; devel at open-fcoe.org
> >Cc: open-iscsi at googlegroups.com
> >Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> restructuring
> >
> >> -----Original Message-----
> >> From: devel-bounces at open-fcoe.org [mailto:devel-bounces at open-
> fcoe.org]
> >> On Behalf Of Eric Multanen
> >> Sent: Tuesday, December 15, 2009 10:31 AM
> >> To: devel at open-fcoe.org
> >> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
restructuring
> >>
> >> v2 - restore code in the service script which destroys FCoE
> interfaces
> >> for
> >> which DCB is not required when the service is stopped.
> >>
> >> This patch set provides a fairly significant redesign of fcoemon.
> >> fcoemon
> >> was suffering from a number of issues, including being too reactive
> to
> >> events.  This resulted in unstable behavior.  This note describes
> the
> >> high
> >> level functions of the redesigned fcoemon.  The following patches
> >> provide the
> >> implementation.
> >>
> >> fcoemon will now have the following high level structure:
> >> 1.  Read in FCoE interface configuration files.  This information
is
> >> used
> >>     to create a list of FCoE interfaces.  As link and dcbd events
> >>     occur and dcbd queries are resolved, the FCoE interface list
> will
> >> manage
> >>     the creation and destruction of FCoE interfaces.  Each FCoE
> >> interface
> >>     element in the list keeps track of its network interface (from
> the
> >>     config file and could be a VLAN), its real network interface
> >(found
> >>     from link events during runtime), and its next action (create,
> >> destroy,
> >>     etc.).
> >>
> >> 2.  As link events are received from the system, fcoemon determines
> >> which ones
> >>     are relevent for the FCoE interfaces.  Relevent link events are
> >> those which
> >>     occur on interfaces which are configured to be FCoE interfaces
> >(see
> >>     the FCoE interface list above), or on the underlying real
> network
> >> interface,
> >>     if the FCoE interface is configured on a VLAN.  A list of
> relevant
> >> real
> >>     network interfaces is maintained.  The list is used to track
the
> >>     operstate of the real network interfaces and, if DCB is
required,
> >>     the status of dcbd queries for the real network interface.
> >>
> >> 3.  The core loop of fcoemon:
> >> 	- listens for link events from rtnetlink and updates the state
> >of
> >> the
> >> 	  real network interface list. Link down events will terminate
> >> any
> >> 	  pending dcbd query sequences and a subsequent link up will
> >> start the
> >> 	  sequence over from the beginning.  Delete link events (VLAN
> >> deleted,
> >> 	  driver unloaded) will mark the FCoE interface for destruction.
> >> 	- listens for dcbd events and response messages.  If the
> >response
> >> 	  matches the current dcbd query state, then move to the next
> >> state.
> >> 	  If a dcbd event indicates a change in a DCB feature of
> >interest,
> >> 	  then start the dcbd query sequence over.
> >> 	- manages a timeout for maintaining a connection to dcbd.  If
> >the
> >> dcbd
> >> 	  service stops, this timeout will clean up as needed and re-
> >> establish
> >> 	  the dcbd connection once dcbd starts up again.
> >> 	- at the end of each event/timeout cycle, after all link and
> >dcbd
> >> 	  and network interface events have been handled, perform any
> >> 	  pending next actions.
> >> 	  For network interface elements, that could mean sending the
> >> next
> >> 	  dcbd query, or, if the dcbd queries are complete, analyze the
> >> 	  results and determine if the FCoE interface should be
> >> 	  will now handle all FCoE interfaces whether or not DCB is
> >> required.
> >> 	  For FCoE interfaces, this means creating or destroying the
> >FCoE
> >> 	  interface.
> >>
> >> 4.  All FCoE interfaces, whether DCB is required or not, will be
> >> managed by
> >>     fcoemon now.  Previously, the init.d start script would handle
> >> configured
> >>     FCoE interfaces which were configured without DCB required.
> >>
> >>
> >> dcbd query sequence
> >> ===================
> >> When DCB is required for an FCoE interface, the real network device
> >> performs a
> >> series of dcbd queries to obtain the current DCB configuration.
The
> >> sequence
> >> is as follows:
> >> 1.  DCB state - is DCB enabled on the interface?
> >> 2.  PFC configuration (Priority Flow Control)
> >> 3.  FCoE configuration
> >> 4.  PFC operational state
> >> 5.  FCoE operational state
> >>
> >> If an error occurs at any step, or if all the steps are completed
> and
> >> the
> >> dcbd state does not match the conditions to either create or
destroy
> >> the
> >> FCoE interface (see next sections), then a retry timer is started.
> >> Once the
> >> timer expires, the dcbd query sequence is reinitiated from the
> >> beginning.  The
> >> timer begins at 0.2 seconds and increases by multiples of that
> amount
> >> up to
> >> ten retries.  Once the max retries has been reached, without a
> >> successful
> >> completion of the dcbd query sequence, then the FCoE interface is
> >> marked for
> >> destruction.
> >>
> >> The function of this retry mechanism is to protect against
> destroying
> >> the
> >> FCoE interface due to intermittent problems.  Such as when links go
> >> down
> >> momentarily due to configuration changes.  When this occurs, dcbd
> may
> >> take
> >> a couple seconds to resynchronize with the peer device.
> >>
> >>
> >> Required conditions to create an FCoE interface
> >> ===============================================
> >> 1.  Link is up on the network interfaces required for the FCoE
> >> interface, and
> >>
> >> 2a. DCB is not required.
> >> 	OR
> >> 2b. DCB is required
> >> 	AND DCB is enabled
> >> 	AND PFC is enabled
> >> 	AND App:FCoE is enabled
> >> 	AND PFC and App:FCoE are operational
> >> 	AND PFC and App:FCoE operational values match
> >>
> >>
> >> Required conditions to destroy an FCoE interface
> >> ================================================
> >> 1.  The network interface required for the FCoE interface is
removed
> >> 	Such as driver is unloaded or VLAN interface is destroyed.
> >> 	This is detected by DELLINK rtnetlink events.
> >>
> >> 2.  The network interfaces required for the FCoE interface are up,
> >> 	AND DCB is required,
> >> 	AND on root network interface
> >> 		DCB is disabled
> >> 			OR
> >> 		App:FCoE is disabled
> >> 			OR
> >> 		PFC and App:FCoE are operational
> >> 		AND PFC and App:FCoE operational values mismatch
> >>
> >> 3.  When DCB is required and the conditions to create the FCoE
> >> interface are
> >>     not satisfied after going through the maximum number of retry
> >> sequences,
> >>     as described in the "dcbd query sequence" section, then the
FCoE
> >> interface
> >>     will be destroyed.
> >>
> >> ---
> >>
> >> Eric Multanen (11):
> >>       Update version string of fcoemon.
> >>       Remove FCoE interface management from service start script
> >>       Add dcbd query timeout and retry mechanism
> >>       Update the fcoemon state machine to be insensitive to
> >> intermittent dcb changes
> >>       Update the FCoE and network interface lists.
> >>       Add separate arguments to specify FCoE and network interface
> >>       Fix the print_errors() routine
> >>       Remove extraneous dcb members from the network interface
> >> structure.
> >>       Fixup dcbd connection timeout code
> >>       Remove unused dcbd routine.
> >>       Obtain the real device of a VLAN interface using an ioctl.
> >>
> >>
> >>  etc/initd/initd.fedora |   66 --
> >>  etc/initd/initd.suse   |   37 -
> >>  fcoemon.c              | 1504
> >++++++++++++++++++++++------------------
> >> --------
> >>  fcoemon.h              |   67 +-
> >>  fcoeplumb.in           |  121 ++--
> >>  5 files changed, 789 insertions(+), 1006 deletions(-)
> >>
> >> --
> >> Signature: Eric Multanen <eric.w.multanen at intel.com>
> >> _______________________________________________
> >> devel mailing list
> >> devel at open-fcoe.org
> >> http://www.open-fcoe.org/mailman/listinfo/devel
> >
> >Shouldn't the requirement to administer DCB  be FCoE independent ?
> After
> >all DCB is required for other protocols like iscsi as well.
> >
> >How about using a library interface to manage DCB over a network
> >interface so that both open-iscsi and open-fcoe could administer DCB?
> 
> dcbd is an independent process and can be administered by client
> programs (e.g. dcbtool).
> fcoemon is also a dcbd client, but currently only queries DCB
> configuration and status.  It
> does not actively administer the DCB configuration.  open-iscsi could
> become a client of dcbd
> as well if there is a need.

Right. But that means makefile kludges and rewrite/rework of a lot of
similar interface specific apis.

By librarifying dcb package open-fcoe/open-iscsi spec files can take
care of dependencies like these and not the installer scripts/makefiles
..




More information about the devel mailing list