[Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring

Multanen, Eric W eric.w.multanen at intel.com
Tue Dec 15 23:38:03 UTC 2009


>-----Original Message-----
>From: Shyam_Iyer at Dell.com [mailto:Shyam_Iyer at Dell.com]
>Sent: Tuesday, December 15, 2009 10:21 AM
>To: Multanen, Eric W; devel at open-fcoe.org
>Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring
>
>> -----Original Message-----
>> From: Multanen, Eric W [mailto:eric.w.multanen at intel.com]
>> Sent: Tuesday, December 15, 2009 11:42 PM
>> To: Iyer, Shyam; devel at open-fcoe.org
>> Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
>> restructuring
>>
>> >-----Original Message-----
>> >From: Shyam_Iyer at Dell.com [mailto:Shyam_Iyer at Dell.com]
>> >Sent: Tuesday, December 15, 2009 9:35 AM
>> >To: Multanen, Eric W; devel at open-fcoe.org
>> >Cc: open-iscsi at googlegroups.com
>> >Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
>> restructuring
>> >
>> >> -----Original Message-----
>> >> From: devel-bounces at open-fcoe.org [mailto:devel-bounces at open-
>> fcoe.org]
>> >> On Behalf Of Eric Multanen
>> >> Sent: Tuesday, December 15, 2009 10:31 AM
>> >> To: devel at open-fcoe.org
>> >> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
>restructuring
>> >>
>> >> v2 - restore code in the service script which destroys FCoE
>> interfaces
>> >> for
>> >> which DCB is not required when the service is stopped.
>> >>
>> >> This patch set provides a fairly significant redesign of fcoemon.
>> >> fcoemon
>> >> was suffering from a number of issues, including being too reactive
>> to
>> >> events.  This resulted in unstable behavior.  This note describes
>> the
>> >> high
>> >> level functions of the redesigned fcoemon.  The following patches
>> >> provide the
>> >> implementation.
>> >>
>> >> fcoemon will now have the following high level structure:
>> >> 1.  Read in FCoE interface configuration files.  This information
>is
>> >> used
>> >>     to create a list of FCoE interfaces.  As link and dcbd events
>> >>     occur and dcbd queries are resolved, the FCoE interface list
>> will
>> >> manage
>> >>     the creation and destruction of FCoE interfaces.  Each FCoE
>> >> interface
>> >>     element in the list keeps track of its network interface (from
>> the
>> >>     config file and could be a VLAN), its real network interface
>> >(found
>> >>     from link events during runtime), and its next action (create,
>> >> destroy,
>> >>     etc.).
>> >>
>> >> 2.  As link events are received from the system, fcoemon determines
>> >> which ones
>> >>     are relevent for the FCoE interfaces.  Relevent link events are
>> >> those which
>> >>     occur on interfaces which are configured to be FCoE interfaces
>> >(see
>> >>     the FCoE interface list above), or on the underlying real
>> network
>> >> interface,
>> >>     if the FCoE interface is configured on a VLAN.  A list of
>> relevant
>> >> real
>> >>     network interfaces is maintained.  The list is used to track
>the
>> >>     operstate of the real network interfaces and, if DCB is
>required,
>> >>     the status of dcbd queries for the real network interface.
>> >>
>> >> 3.  The core loop of fcoemon:
>> >> 	- listens for link events from rtnetlink and updates the state
>> >of
>> >> the
>> >> 	  real network interface list. Link down events will terminate
>> >> any
>> >> 	  pending dcbd query sequences and a subsequent link up will
>> >> start the
>> >> 	  sequence over from the beginning.  Delete link events (VLAN
>> >> deleted,
>> >> 	  driver unloaded) will mark the FCoE interface for destruction.
>> >> 	- listens for dcbd events and response messages.  If the
>> >response
>> >> 	  matches the current dcbd query state, then move to the next
>> >> state.
>> >> 	  If a dcbd event indicates a change in a DCB feature of
>> >interest,
>> >> 	  then start the dcbd query sequence over.
>> >> 	- manages a timeout for maintaining a connection to dcbd.  If
>> >the
>> >> dcbd
>> >> 	  service stops, this timeout will clean up as needed and re-
>> >> establish
>> >> 	  the dcbd connection once dcbd starts up again.
>> >> 	- at the end of each event/timeout cycle, after all link and
>> >dcbd
>> >> 	  and network interface events have been handled, perform any
>> >> 	  pending next actions.
>> >> 	  For network interface elements, that could mean sending the
>> >> next
>> >> 	  dcbd query, or, if the dcbd queries are complete, analyze the
>> >> 	  results and determine if the FCoE interface should be
>> >> 	  will now handle all FCoE interfaces whether or not DCB is
>> >> required.
>> >> 	  For FCoE interfaces, this means creating or destroying the
>> >FCoE
>> >> 	  interface.
>> >>
>> >> 4.  All FCoE interfaces, whether DCB is required or not, will be
>> >> managed by
>> >>     fcoemon now.  Previously, the init.d start script would handle
>> >> configured
>> >>     FCoE interfaces which were configured without DCB required.
>> >>
>> >>
>> >> dcbd query sequence
>> >> ===================
>> >> When DCB is required for an FCoE interface, the real network device
>> >> performs a
>> >> series of dcbd queries to obtain the current DCB configuration.
>The
>> >> sequence
>> >> is as follows:
>> >> 1.  DCB state - is DCB enabled on the interface?
>> >> 2.  PFC configuration (Priority Flow Control)
>> >> 3.  FCoE configuration
>> >> 4.  PFC operational state
>> >> 5.  FCoE operational state
>> >>
>> >> If an error occurs at any step, or if all the steps are completed
>> and
>> >> the
>> >> dcbd state does not match the conditions to either create or
>destroy
>> >> the
>> >> FCoE interface (see next sections), then a retry timer is started.
>> >> Once the
>> >> timer expires, the dcbd query sequence is reinitiated from the
>> >> beginning.  The
>> >> timer begins at 0.2 seconds and increases by multiples of that
>> amount
>> >> up to
>> >> ten retries.  Once the max retries has been reached, without a
>> >> successful
>> >> completion of the dcbd query sequence, then the FCoE interface is
>> >> marked for
>> >> destruction.
>> >>
>> >> The function of this retry mechanism is to protect against
>> destroying
>> >> the
>> >> FCoE interface due to intermittent problems.  Such as when links go
>> >> down
>> >> momentarily due to configuration changes.  When this occurs, dcbd
>> may
>> >> take
>> >> a couple seconds to resynchronize with the peer device.
>> >>
>> >>
>> >> Required conditions to create an FCoE interface
>> >> ===============================================
>> >> 1.  Link is up on the network interfaces required for the FCoE
>> >> interface, and
>> >>
>> >> 2a. DCB is not required.
>> >> 	OR
>> >> 2b. DCB is required
>> >> 	AND DCB is enabled
>> >> 	AND PFC is enabled
>> >> 	AND App:FCoE is enabled
>> >> 	AND PFC and App:FCoE are operational
>> >> 	AND PFC and App:FCoE operational values match
>> >>
>> >>
>> >> Required conditions to destroy an FCoE interface
>> >> ================================================
>> >> 1.  The network interface required for the FCoE interface is
>removed
>> >> 	Such as driver is unloaded or VLAN interface is destroyed.
>> >> 	This is detected by DELLINK rtnetlink events.
>> >>
>> >> 2.  The network interfaces required for the FCoE interface are up,
>> >> 	AND DCB is required,
>> >> 	AND on root network interface
>> >> 		DCB is disabled
>> >> 			OR
>> >> 		App:FCoE is disabled
>> >> 			OR
>> >> 		PFC and App:FCoE are operational
>> >> 		AND PFC and App:FCoE operational values mismatch
>> >>
>> >> 3.  When DCB is required and the conditions to create the FCoE
>> >> interface are
>> >>     not satisfied after going through the maximum number of retry
>> >> sequences,
>> >>     as described in the "dcbd query sequence" section, then the
>FCoE
>> >> interface
>> >>     will be destroyed.
>> >>
>> >> ---
>> >>
>> >> Eric Multanen (11):
>> >>       Update version string of fcoemon.
>> >>       Remove FCoE interface management from service start script
>> >>       Add dcbd query timeout and retry mechanism
>> >>       Update the fcoemon state machine to be insensitive to
>> >> intermittent dcb changes
>> >>       Update the FCoE and network interface lists.
>> >>       Add separate arguments to specify FCoE and network interface
>> >>       Fix the print_errors() routine
>> >>       Remove extraneous dcb members from the network interface
>> >> structure.
>> >>       Fixup dcbd connection timeout code
>> >>       Remove unused dcbd routine.
>> >>       Obtain the real device of a VLAN interface using an ioctl.
>> >>
>> >>
>> >>  etc/initd/initd.fedora |   66 --
>> >>  etc/initd/initd.suse   |   37 -
>> >>  fcoemon.c              | 1504
>> >++++++++++++++++++++++------------------
>> >> --------
>> >>  fcoemon.h              |   67 +-
>> >>  fcoeplumb.in           |  121 ++--
>> >>  5 files changed, 789 insertions(+), 1006 deletions(-)
>> >>
>> >> --
>> >> Signature: Eric Multanen <eric.w.multanen at intel.com>
>> >> _______________________________________________
>> >> devel mailing list
>> >> devel at open-fcoe.org
>> >> http://www.open-fcoe.org/mailman/listinfo/devel
>> >
>> >Shouldn't the requirement to administer DCB  be FCoE independent ?
>> After
>> >all DCB is required for other protocols like iscsi as well.
>> >
>> >How about using a library interface to manage DCB over a network
>> >interface so that both open-iscsi and open-fcoe could administer DCB?
>>
>> dcbd is an independent process and can be administered by client
>> programs (e.g. dcbtool).
>> fcoemon is also a dcbd client, but currently only queries DCB
>> configuration and status.  It
>> does not actively administer the DCB configuration.  open-iscsi could
>> become a client of dcbd
>> as well if there is a need.
>
>Right. But that means makefile kludges and rewrite/rework of a lot of
>similar interface specific apis.
>
>By librarifying dcb package open-fcoe/open-iscsi spec files can take
>care of dependencies like these and not the installer scripts/makefiles
>..

I think you are proposing a change which is beyond the scope of what this
patch series is trying to accomplish.  It sounds like a good idea, but what this
patch series is doing is improving fcoemon's pre-existing methods for monitoring
and acting upon network and dcbd events.








More information about the devel mailing list