[Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring

Shyam_Iyer at Dell.com Shyam_Iyer at Dell.com
Wed Dec 16 09:37:10 UTC 2009


> -----Original Message-----
> From: devel-bounces at open-fcoe.org [mailto:devel-bounces at open-fcoe.org]
> On Behalf Of Multanen, Eric W
> Sent: Wednesday, December 16, 2009 5:08 AM
> To: Iyer, Shyam; devel at open-fcoe.org
> Subject: Re: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> restructuring
> 
> >-----Original Message-----
> >From: Shyam_Iyer at Dell.com [mailto:Shyam_Iyer at Dell.com]
> >Sent: Tuesday, December 15, 2009 10:21 AM
> >To: Multanen, Eric W; devel at open-fcoe.org
> >Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> restructuring
> >
> >> -----Original Message-----
> >> From: Multanen, Eric W [mailto:eric.w.multanen at intel.com]
> >> Sent: Tuesday, December 15, 2009 11:42 PM
> >> To: Iyer, Shyam; devel at open-fcoe.org
> >> Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> >> restructuring
> >>
> >> >-----Original Message-----
> >> >From: Shyam_Iyer at Dell.com [mailto:Shyam_Iyer at Dell.com]
> >> >Sent: Tuesday, December 15, 2009 9:35 AM
> >> >To: Multanen, Eric W; devel at open-fcoe.org
> >> >Cc: open-iscsi at googlegroups.com
> >> >Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> >> restructuring
> >> >
> >> >> -----Original Message-----
> >> >> From: devel-bounces at open-fcoe.org [mailto:devel-bounces at open-
> >> fcoe.org]
> >> >> On Behalf Of Eric Multanen
> >> >> Sent: Tuesday, December 15, 2009 10:31 AM
> >> >> To: devel at open-fcoe.org
> >> >> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> >restructuring
> >> >>
> >> >> v2 - restore code in the service script which destroys FCoE
> >> interfaces
> >> >> for
> >> >> which DCB is not required when the service is stopped.
> >> >>
> >> >> This patch set provides a fairly significant redesign of
fcoemon.
> >> >> fcoemon
> >> >> was suffering from a number of issues, including being too
> reactive
> >> to
> >> >> events.  This resulted in unstable behavior.  This note
describes
> >> the
> >> >> high
> >> >> level functions of the redesigned fcoemon.  The following
patches
> >> >> provide the
> >> >> implementation.
> >> >>
> >> >> fcoemon will now have the following high level structure:
> >> >> 1.  Read in FCoE interface configuration files.  This
information
> >is
> >> >> used
> >> >>     to create a list of FCoE interfaces.  As link and dcbd
events
> >> >>     occur and dcbd queries are resolved, the FCoE interface list
> >> will
> >> >> manage
> >> >>     the creation and destruction of FCoE interfaces.  Each FCoE
> >> >> interface
> >> >>     element in the list keeps track of its network interface
> (from
> >> the
> >> >>     config file and could be a VLAN), its real network interface
> >> >(found
> >> >>     from link events during runtime), and its next action
(create,
> >> >> destroy,
> >> >>     etc.).
> >> >>
> >> >> 2.  As link events are received from the system, fcoemon
> determines
> >> >> which ones
> >> >>     are relevent for the FCoE interfaces.  Relevent link events
> are
> >> >> those which
> >> >>     occur on interfaces which are configured to be FCoE
> interfaces
> >> >(see
> >> >>     the FCoE interface list above), or on the underlying real
> >> network
> >> >> interface,
> >> >>     if the FCoE interface is configured on a VLAN.  A list of
> >> relevant
> >> >> real
> >> >>     network interfaces is maintained.  The list is used to track
> >the
> >> >>     operstate of the real network interfaces and, if DCB is
> >required,
> >> >>     the status of dcbd queries for the real network interface.
> >> >>
> >> >> 3.  The core loop of fcoemon:
> >> >> 	- listens for link events from rtnetlink and updates the
> state
> >> >of
> >> >> the
> >> >> 	  real network interface list. Link down events will
> terminate
> >> >> any
> >> >> 	  pending dcbd query sequences and a subsequent link up
> will
> >> >> start the
> >> >> 	  sequence over from the beginning.  Delete link events
> (VLAN
> >> >> deleted,
> >> >> 	  driver unloaded) will mark the FCoE interface for
> destruction.
> >> >> 	- listens for dcbd events and response messages.  If the
> >> >response
> >> >> 	  matches the current dcbd query state, then move to the
> next
> >> >> state.
> >> >> 	  If a dcbd event indicates a change in a DCB feature of
> >> >interest,
> >> >> 	  then start the dcbd query sequence over.
> >> >> 	- manages a timeout for maintaining a connection to
dcbd.
> If
> >> >the
> >> >> dcbd
> >> >> 	  service stops, this timeout will clean up as needed
and
> re-
> >> >> establish
> >> >> 	  the dcbd connection once dcbd starts up again.
> >> >> 	- at the end of each event/timeout cycle, after all link
> and
> >> >dcbd
> >> >> 	  and network interface events have been handled,
perform
> any
> >> >> 	  pending next actions.
> >> >> 	  For network interface elements, that could mean
sending
> the
> >> >> next
> >> >> 	  dcbd query, or, if the dcbd queries are complete,
analyze
> the
> >> >> 	  results and determine if the FCoE interface should be
> >> >> 	  will now handle all FCoE interfaces whether or not DCB
is
> >> >> required.
> >> >> 	  For FCoE interfaces, this means creating or destroying
> the
> >> >FCoE
> >> >> 	  interface.
> >> >>
> >> >> 4.  All FCoE interfaces, whether DCB is required or not, will be
> >> >> managed by
> >> >>     fcoemon now.  Previously, the init.d start script would
> handle
> >> >> configured
> >> >>     FCoE interfaces which were configured without DCB required.
> >> >>
> >> >>
> >> >> dcbd query sequence
> >> >> ===================
> >> >> When DCB is required for an FCoE interface, the real network
> device
> >> >> performs a
> >> >> series of dcbd queries to obtain the current DCB configuration.
> >The
> >> >> sequence
> >> >> is as follows:
> >> >> 1.  DCB state - is DCB enabled on the interface?
> >> >> 2.  PFC configuration (Priority Flow Control)
> >> >> 3.  FCoE configuration
> >> >> 4.  PFC operational state
> >> >> 5.  FCoE operational state
> >> >>
> >> >> If an error occurs at any step, or if all the steps are
completed
> >> and
> >> >> the
> >> >> dcbd state does not match the conditions to either create or
> >destroy
> >> >> the
> >> >> FCoE interface (see next sections), then a retry timer is
started.
> >> >> Once the
> >> >> timer expires, the dcbd query sequence is reinitiated from the
> >> >> beginning.  The
> >> >> timer begins at 0.2 seconds and increases by multiples of that
> >> amount
> >> >> up to
> >> >> ten retries.  Once the max retries has been reached, without a
> >> >> successful
> >> >> completion of the dcbd query sequence, then the FCoE interface
is
> >> >> marked for
> >> >> destruction.
> >> >>
> >> >> The function of this retry mechanism is to protect against
> >> destroying
> >> >> the
> >> >> FCoE interface due to intermittent problems.  Such as when links
> go
> >> >> down
> >> >> momentarily due to configuration changes.  When this occurs,
dcbd
> >> may
> >> >> take
> >> >> a couple seconds to resynchronize with the peer device.
> >> >>
> >> >>
> >> >> Required conditions to create an FCoE interface
> >> >> ===============================================
> >> >> 1.  Link is up on the network interfaces required for the FCoE
> >> >> interface, and
> >> >>
> >> >> 2a. DCB is not required.
> >> >> 	OR
> >> >> 2b. DCB is required
> >> >> 	AND DCB is enabled
> >> >> 	AND PFC is enabled
> >> >> 	AND App:FCoE is enabled
> >> >> 	AND PFC and App:FCoE are operational
> >> >> 	AND PFC and App:FCoE operational values match
> >> >>
> >> >>
> >> >> Required conditions to destroy an FCoE interface
> >> >> ================================================
> >> >> 1.  The network interface required for the FCoE interface is
> >removed
> >> >> 	Such as driver is unloaded or VLAN interface is
destroyed.
> >> >> 	This is detected by DELLINK rtnetlink events.
> >> >>
> >> >> 2.  The network interfaces required for the FCoE interface are
up,
> >> >> 	AND DCB is required,
> >> >> 	AND on root network interface
> >> >> 		DCB is disabled
> >> >> 			OR
> >> >> 		App:FCoE is disabled
> >> >> 			OR
> >> >> 		PFC and App:FCoE are operational
> >> >> 		AND PFC and App:FCoE operational values mismatch
> >> >>
> >> >> 3.  When DCB is required and the conditions to create the FCoE
> >> >> interface are
> >> >>     not satisfied after going through the maximum number of
retry
> >> >> sequences,
> >> >>     as described in the "dcbd query sequence" section, then the
> >FCoE
> >> >> interface
> >> >>     will be destroyed.
> >> >>
> >> >> ---
> >> >>
> >> >> Eric Multanen (11):
> >> >>       Update version string of fcoemon.
> >> >>       Remove FCoE interface management from service start script
> >> >>       Add dcbd query timeout and retry mechanism
> >> >>       Update the fcoemon state machine to be insensitive to
> >> >> intermittent dcb changes
> >> >>       Update the FCoE and network interface lists.
> >> >>       Add separate arguments to specify FCoE and network
> interface
> >> >>       Fix the print_errors() routine
> >> >>       Remove extraneous dcb members from the network interface
> >> >> structure.
> >> >>       Fixup dcbd connection timeout code
> >> >>       Remove unused dcbd routine.
> >> >>       Obtain the real device of a VLAN interface using an ioctl.
> >> >>
> >> >>
> >> >>  etc/initd/initd.fedora |   66 --
> >> >>  etc/initd/initd.suse   |   37 -
> >> >>  fcoemon.c              | 1504
> >> >++++++++++++++++++++++------------------
> >> >> --------
> >> >>  fcoemon.h              |   67 +-
> >> >>  fcoeplumb.in           |  121 ++--
> >> >>  5 files changed, 789 insertions(+), 1006 deletions(-)
> >> >>
> >> >> --
> >> >> Signature: Eric Multanen <eric.w.multanen at intel.com>
> >> >> _______________________________________________
> >> >> devel mailing list
> >> >> devel at open-fcoe.org
> >> >> http://www.open-fcoe.org/mailman/listinfo/devel
> >> >
> >> >Shouldn't the requirement to administer DCB  be FCoE independent ?
> >> After
> >> >all DCB is required for other protocols like iscsi as well.
> >> >
> >> >How about using a library interface to manage DCB over a network
> >> >interface so that both open-iscsi and open-fcoe could administer
> DCB?
> >>
> >> dcbd is an independent process and can be administered by client
> >> programs (e.g. dcbtool).
> >> fcoemon is also a dcbd client, but currently only queries DCB
> >> configuration and status.  It
> >> does not actively administer the DCB configuration.  open-iscsi
> could
> >> become a client of dcbd
> >> as well if there is a need.
> >
> >Right. But that means makefile kludges and rewrite/rework of a lot of
> >similar interface specific apis.
> >
> >By librarifying dcb package open-fcoe/open-iscsi spec files can take
> >care of dependencies like these and not the installer
> scripts/makefiles
> >..
> 
> I think you are proposing a change which is beyond the scope of what
> this
> patch series is trying to accomplish.  It sounds like a good idea, but
> what this
> patch series is doing is improving fcoemon's pre-existing methods for
> monitoring
> and acting upon network and dcbd events.

Agree. Sorry for picking on this patch series. 
But since I read the commit log talk about redesigning interface with
DCB, I thought this might be worth considering now.



More information about the devel mailing list