520 lines
15 KiB
C
520 lines
15 KiB
C
/* fmanKgSmpDemo.c - SMP test code for exercising Freescale QorIQ Fman key generator */
|
|
|
|
/*
|
|
* Copyright (c) 2009 Wind River Systems, Inc.
|
|
*
|
|
* The right to copy, distribute, modify or otherwise make use
|
|
* of this software may be licensed only pursuant to the terms
|
|
* of an applicable Wind River license agreement.
|
|
*/
|
|
|
|
/*
|
|
modification history
|
|
--------------------
|
|
01d,24feb10,wap Remove extra declaration of vxCpuIdGet()
|
|
01c,20jan10,wap Use FMAN_PKT_DATA() and FMAN_PKT_IFC() macros
|
|
01b,27nov09,wap Minor cleanups
|
|
01a,22oct09,wap written
|
|
*/
|
|
|
|
/*
|
|
DESCRIPTION
|
|
|
|
This module contains demo code for exercising the key generator feature of
|
|
the Freescale QorIQ Frame Manager in an SMP environment. The key generator
|
|
can extract a specified portion of an incoming frame and calculate a CRC64
|
|
hash of it in hardware. The extracted key and the hash are provided in the
|
|
internal frame context data for a frame, and can be provided to the host
|
|
CPU as a header prepended to frame data.
|
|
|
|
This demo is meant to illustrate how to take advantage of the P4080's multicore
|
|
and network acceleration features in an SMP context. A more common method is
|
|
to use an AMP configuration instead. In that scenario, there are multiple OS
|
|
contexts running on the P4080 at the same time. One context is dedicated
|
|
entirely for management, and is responsible for configuring the DTSEC ports
|
|
and frame manager. This context can install key generator schemes or classifier
|
|
rules to divert incoming frames to various cores. Meanwhile, additional OS
|
|
contexts running on those cores (e.g. AMP VxWorks instances) can perform
|
|
additional processing on the frame data. Several cores can be arranged in
|
|
stages to perform various tasks, before finally sending the frame to an egress
|
|
port.
|
|
|
|
In SMP, the principle is similar, except there is only one OS context, namely
|
|
an SMP instance of VxWorks. The VxWorks scheduler is used to dispatch tasks onto
|
|
various cores controlled by VxWorks/SMP, and through the use of CPU affinity
|
|
selection, those tasks remain executing on those cores exclusively. Each task
|
|
interacts only with the buffer manager and queue manager portals assigned to
|
|
its particular core, and runs in a busy loop without blocking. This prevents
|
|
each task from being scheduled off of the core, and avoiding the use of software
|
|
locking such as spinlocks improves parallelization (and hence overall
|
|
performance).
|
|
|
|
This demo provides an example of this using the frame manager key generator.
|
|
The demo is run with a VxWorks SMP image, configured to use dtsec0. A key
|
|
generator scheme is installed that extracts the IPv4 source address and hashes
|
|
it. This is similar to the fmanKgDemo code. The difference is how the frames
|
|
are processed. The demo code sets its CPU affinity so that it becomes bound
|
|
to a specified core (the P4080 has 8 CPUs), and once there it disables the
|
|
normal interrupt driven event handling of the queue manager. Instead, the demo
|
|
task runs in a busy loop on the core, checking for work to dequeue itself. When
|
|
a frame arrives, it is intercepted by the processing loop running on core 7,
|
|
which prints some info about the frame and then requeues it for delivery to
|
|
the DTSEC driver's normal RX frame queue.
|
|
|
|
This particular approach is useful for implementing a fast forwarding framework
|
|
that uses both hardware acceleration and multiple processors to improve
|
|
performance:
|
|
|
|
- The key generator speeds up packet processing by calculating hashes of header
|
|
fields (e.g. source and destination addresses) in hardware
|
|
- The queue manager allows work to be dispatched between cores or hardware
|
|
modules without requiring software locking or waiting
|
|
- Using multiple CPUs allows management tasks to be executed on some cores
|
|
while dedicated processing tasks make exclusive use of others
|
|
|
|
In a real fast forwarding application, the processing loop could use the hash
|
|
values to do fast routing decision lookups, or could return the frame to the
|
|
DTSEC for slow path processing. A similar demo could also be constructed using
|
|
the coarse classifier and the key generator working together.
|
|
|
|
The demo also calculates the hash of the key in software to illustrate
|
|
that software can generate its own hashes that match those produced
|
|
by the hardware.
|
|
|
|
Currently, the demo expects to use the first DTSEC port on the first Fman
|
|
instance (dtsec0).
|
|
|
|
The demo can be executed as follows:
|
|
|
|
- Create a Vxworks SMP project, using the simP4080 BSP. This
|
|
requires that the user have VxWorks SMP installed.
|
|
- Ensure the following components are present in the project:
|
|
INCLUDE_SHELL
|
|
DRV_RESOURCE_QORIQBMAN
|
|
DRV_RESOURCE_QORIQLAW
|
|
DRV_RESOURCE_QORIQQMAN
|
|
DRV_RESOURCE_QORQIFMAN
|
|
DRV_VXBEND_DTSEC
|
|
DRV_MII_DTSEC_MDIO
|
|
DRV_RESOURCE_FSLGPIO
|
|
- Add the fmanKgSmpDemo.c file to the project
|
|
- Boot VxWorks
|
|
- Run the test as follows:
|
|
|
|
-> sp fmanKgSmpDemoStart, 0x123, 7
|
|
|
|
This will spawn the demo into a task, which will then bind itself to
|
|
CPU 7, install a key generator scheme, and enable key generator handling
|
|
on the network port. The demo will continue executing on CPU 7 until
|
|
terminated. The following output will be observed:
|
|
|
|
-> sp fmanKgSmpDemoStart, 0x123, 7
|
|
Task spawned: id = 0x3fcff630, name = t2
|
|
value = 1070593584 = 0x3fcff630 = '0'
|
|
-> -----
|
|
got frame 0x4871dc2, length: 60
|
|
keygen key length: 4
|
|
keygen hash: [7F37124AE12C93DB]
|
|
software hash: [7F37124AE12C93DB]
|
|
keygen key: [930BD956]
|
|
-----
|
|
-----
|
|
got frame 0x4871dc2, length: 114
|
|
keygen key length: 4
|
|
keygen hash: [7F37124AE12C93DB]
|
|
software hash: [7F37124AE12C93DB]
|
|
keygen key: [930BD956]
|
|
-----
|
|
-----
|
|
got frame 0x4871dc2, length: 98
|
|
keygen key length: 4
|
|
keygen hash: [24DE31B309D9A940]
|
|
software hash: [24DE31B309D9A940]
|
|
keygen key: [930BD91B]
|
|
-----
|
|
-----
|
|
got frame 0x4871dc2, length: 60
|
|
keygen key length: 4
|
|
keygen hash: [B110F3DD0AF30A6F]
|
|
software hash: [B110F3DD0AF30A6F]
|
|
keygen key: [12345678]
|
|
-----
|
|
|
|
To terminate the demo, do the following:
|
|
|
|
-> fmanKgSmpDemoStop
|
|
|
|
This will switch the desired port back to normal operation,
|
|
delete the key generator scheme, and terminate the processing
|
|
task. The CPU's queue manager portal will have interrupt driven
|
|
event processing re-enabled.
|
|
|
|
*/
|
|
|
|
#include <vxWorks.h>
|
|
|
|
#include <stdlib.h>
|
|
#include <string.h>
|
|
#include <stdio.h>
|
|
|
|
#include <semLib.h>
|
|
#include <taskLib.h>
|
|
#include <vxBusLib.h>
|
|
#include <cpuset.h>
|
|
#include <hwif/vxbus/vxBus.h>
|
|
#include <hwif/util/vxbParamSys.h>
|
|
#include <hwif/util/vxbDmaBufLib.h>
|
|
|
|
#include <etherMultiLib.h>
|
|
#include <end.h>
|
|
#include <endLib.h>
|
|
|
|
#include <../src/hwif/h/resource/vxbQorIQQmanPortal.h>
|
|
#include <../src/hwif/h/resource/vxbQorIQBmanPortal.h>
|
|
#include <../src/hwif/h/resource/vxbQorIQFman.h>
|
|
#include <../src/hwif/h/end/vxbDtsecEnd.h>
|
|
|
|
#define FMAN_KG_DEMO_FMAN_UNIT 0
|
|
#define FMAN_KG_DEMO_PORT FMAN_PORTID_DTSEC_RX0
|
|
#define FMAN_KG_DEMO_FQID DTSEC_RX_FQID
|
|
|
|
LOCAL UINT32 fmanKgTestInputFqId;
|
|
LOCAL UINT32 fmanKgTestCtxB = 0x123456;
|
|
LOCAL BOOL fmanKgTestQuit = FALSE;
|
|
|
|
/*****************************************************************************
|
|
*
|
|
* processLoop - packet processing loop
|
|
*
|
|
* This function polls the queue manager on the current core for dequeue
|
|
* events generated as the result of arriving ethernet frames. When a frame
|
|
* arrives, it will print some information about it, including the buffer
|
|
* address of the frame, its length, and the fields and hash provided by
|
|
* the key generator hardware. The frame will then be queued up to the queue
|
|
* manager again to be sent to the DTSEC driver.
|
|
*
|
|
* RETURNS: N/A
|
|
*
|
|
* ERRNO: N/A
|
|
*/
|
|
|
|
LOCAL void processLoop
|
|
(
|
|
void
|
|
)
|
|
{
|
|
QPORTAL * portal;
|
|
UINT32 fqId;
|
|
UINT8 bpid;
|
|
UINT32 ctxB;
|
|
QPORTAL_FDESC fd;
|
|
char * virt;
|
|
volatile FMAN_PR * pPr;
|
|
volatile FMAN_IFC * pIfc;
|
|
char * pData;
|
|
int i;
|
|
|
|
/* Get a handle for the portal on this CPU. */
|
|
|
|
qportalAcquire (vxCpuIdGet (), &portal);
|
|
|
|
FOREVER
|
|
{
|
|
|
|
if (fmanKgTestQuit == TRUE)
|
|
break;
|
|
|
|
if (qportalDequeue (portal, &fqId, &ctxB, &fd) != OK)
|
|
continue;
|
|
|
|
if (ctxB != fmanKgTestCtxB)
|
|
continue;
|
|
|
|
bpid = fd.qportalBpid;
|
|
virt = (char *)fd.qportalAddrLo;
|
|
|
|
/* Get a pointer to the internal frame context */
|
|
|
|
pIfc = FMAN_PKT_IFC(virt);
|
|
|
|
/* Get a pointer to the parser result */
|
|
|
|
pPr = &pIfc->fman_ifc_pr;
|
|
|
|
/* Start of ethernet frame data is here. */
|
|
|
|
pData = FMAN_PKT_DATA(virt);
|
|
|
|
printf ("-----\n");
|
|
|
|
printf ("got frame %p, length: %d\n", pData, (fd.qportalOffLen &
|
|
QPORTAL_FDESC_OFFLEN_LENGTH));
|
|
|
|
/* See if the frame has been tagged by the key generator */
|
|
|
|
if (pIfc->fman_ifc_keysz)
|
|
{
|
|
printf ("keygen key length: %d\n", pIfc->fman_ifc_keysz);
|
|
printf ("keygen hash: [%llX]\n", pIfc->fman_ifc_hash);
|
|
printf ("software hash: [%llX]\n",
|
|
fmanKgHash ((UINT8 *)pIfc->fman_ifc_key, pIfc->fman_ifc_keysz));
|
|
printf ("keygen key: [");
|
|
for (i = 0; i < pIfc->fman_ifc_keysz ; i++)
|
|
printf ("%02X", pIfc->fman_ifc_key[i]);
|
|
printf ("]\n");
|
|
}
|
|
|
|
printf ("-----\n");
|
|
|
|
/* Return the buffer to the buffer manager. */
|
|
|
|
qportalEnqueue (portal, fmanKgTestInputFqId, 0, &fd);
|
|
}
|
|
|
|
return;
|
|
}
|
|
|
|
/*****************************************************************************
|
|
*
|
|
* fmanKgSchemeCreate - set up a sample key generator scheme
|
|
*
|
|
* This function initializes a key generator scheme index <schemeIdx>. The
|
|
* scheme uses the 'extract known fields' mask to have the key generator
|
|
* extract the first IP source address from received packets, and then
|
|
* direct the frame to the frame queue specified by <fqid>. The next
|
|
* action field of the 'mode' word in the scheme is set to direct the
|
|
* frame to the 'BMI enqueue' path so that the frame can be enqueued
|
|
* to the queue manager.
|
|
*
|
|
* The scheme can be modified to extract the IP destination address instead
|
|
* of the source, or to extract both fields together.
|
|
*
|
|
* Once the scheme is loaded, the fmanKgPortConfig() routine is used to
|
|
* tie it to a given port. Note that it's possible to associate a given
|
|
* scheme with more than one port.
|
|
*
|
|
* Note that the scheme is only activated by having the FMKG_SE_MODE_SI
|
|
* bit set in the 'mode' word of the scheme. Without this bit, the scheme
|
|
* will still be loaded into the specified <schemeIdx> slot, but it will
|
|
* be ignored by the frame manager.
|
|
*
|
|
* RETURNS: N/A
|
|
*
|
|
* ERRNO: N/A
|
|
*/
|
|
|
|
LOCAL void fmanKgSchemeCreate
|
|
(
|
|
VXB_DEVICE_ID pDev,
|
|
UINT32 fqid,
|
|
int schemeIdx,
|
|
int port
|
|
)
|
|
{
|
|
FMAN_KG_SCHEME sc;
|
|
|
|
if (pDev == NULL)
|
|
return;
|
|
|
|
/*
|
|
* Create a scheme that extracts the IPv4 source address from
|
|
* the first IP header, using the 'extract known fields' mask.
|
|
* Also, specify a default value of 0x12345678; this will be
|
|
* used as the default key for frames where no IP header is
|
|
* present.
|
|
*/
|
|
|
|
bzero ((char *)&sc, sizeof(sc));
|
|
|
|
sc.fmks_mode = FMKG_SE_MODE_SI | FMAN_NIA_BMIEQ;
|
|
sc.fmks_ekfc = FMKG_SE_EKFC_IPSRC1;
|
|
sc.fmks_fqb = fqid;
|
|
sc.fmks_dv0 = 0x12345678;
|
|
sc.fmks_ekdv = FMKG_EKDV_SCHEME0;
|
|
|
|
/* Load the scheme to the desired scheme index */
|
|
|
|
fmanKgSchemeWrite (pDev, schemeIdx, &sc);
|
|
|
|
/* Tie the scheme index to a port */
|
|
|
|
fmanKgPortConfig (pDev, port, schemeIdx);
|
|
|
|
return;
|
|
}
|
|
|
|
/*****************************************************************************
|
|
*
|
|
* fmanKgSchemeDestroy - delete a sample key generator scheme
|
|
*
|
|
* This function deletes the key generator scheme index <schemeIdx>.
|
|
* This is done disabling key generator usage for the specified <port>
|
|
* and then overwriting the specified scheme index slot with a blank
|
|
* scheme structure.
|
|
*
|
|
* RETURNS: N/A
|
|
*
|
|
* ERRNO: N/A
|
|
*/
|
|
|
|
LOCAL void fmanKgSchemeDestroy
|
|
(
|
|
VXB_DEVICE_ID pDev,
|
|
UINT32 fqid,
|
|
int schemeIdx,
|
|
int port
|
|
)
|
|
{
|
|
FMAN_KG_SCHEME sc;
|
|
|
|
/* Find the handle for the first Fman instance. */
|
|
|
|
pDev = vxbInstByNameFind ("QorIQFman", FMAN_KG_DEMO_FMAN_UNIT);
|
|
|
|
if (pDev == NULL)
|
|
return;
|
|
|
|
/* Stop using the key generator on this port */
|
|
|
|
fmanKgPortHalt (pDev, port, schemeIdx);
|
|
|
|
/* Delete the scheme. */
|
|
|
|
bzero ((char *)&sc, sizeof(sc));
|
|
fmanKgSchemeWrite (pDev, schemeIdx, &sc);
|
|
|
|
return;
|
|
}
|
|
|
|
/*****************************************************************************
|
|
*
|
|
* fmanKgSmpDemoStart - start the key generator demo
|
|
*
|
|
* This function starts the key generator demo running using a user-specified
|
|
* <fqid>. This is the frame queue to which all key extracted frames will
|
|
* be delivered, and the cpu <core> number on which the demo code will run.
|
|
* (The demo must be spawned into a separate task to run correctly.) The
|
|
* demo disables interrupt-driven event handling on the specified CPU and
|
|
* creates a frame queue ID targeted to the specified <core>. It then
|
|
* installs a scheme that extracts the source IP address from all frames.
|
|
* Finally, it runs in a busy loop processing all frames that arrive via
|
|
* the frame queue.
|
|
*
|
|
* The scheme and event handler remain in place until fmanKgSmpDemoStop()
|
|
* is called.
|
|
*
|
|
* RETURNS: N/A
|
|
*
|
|
* ERRNO: N/A
|
|
*/
|
|
|
|
void fmanKgSmpDemoStart
|
|
(
|
|
UINT32 fqid,
|
|
int core
|
|
)
|
|
{
|
|
VXB_DEVICE_ID pDev;
|
|
QPORTAL * portal = NULL;
|
|
int cpu = 0;
|
|
cpuset_t cpuSet;
|
|
STATUS r;
|
|
|
|
/* Find the handle for the first Fman instance. */
|
|
|
|
pDev = vxbInstByNameFind ("QorIQFman", FMAN_KG_DEMO_FMAN_UNIT);
|
|
|
|
if (pDev == NULL)
|
|
{
|
|
printf ("failed to fine Fman instance...\n");
|
|
goto err;
|
|
}
|
|
|
|
cpu = core;
|
|
fmanKgTestQuit = FALSE;
|
|
|
|
if (qportalAcquire (cpu, &portal) != OK)
|
|
{
|
|
printf ("failed to acquire portal handle...\n");
|
|
goto err;
|
|
}
|
|
|
|
/* Create a frame queue targeted to the specified CPU core */
|
|
|
|
r = qportalFqInit (portal, fqid, QPORTAL_CHAN_SW0 + cpu, 0,
|
|
fmanKgTestCtxB, 0, 0, 0);
|
|
|
|
if (r != OK)
|
|
{
|
|
printf ("failed to create frame queue...\n");
|
|
qportalRelease (cpu, portal);
|
|
goto err;
|
|
}
|
|
|
|
/* Now bind this task to the specified CPU buy setting affinity. */
|
|
|
|
CPUSET_ZERO(cpuSet);
|
|
CPUSET_SET(cpuSet, core);
|
|
taskCpuAffinitySet (taskIdSelf (), cpuSet);
|
|
|
|
/*
|
|
* Disable interrupt driven event handling for this portal as
|
|
* we'll be using our own dispatch look to handle events.
|
|
*/
|
|
|
|
qportalIntHandlingDisable (portal);
|
|
|
|
/*
|
|
* The VxWorks DTSEC driver uses RX frame queue
|
|
* IDs with a base of 0x10.
|
|
*/
|
|
|
|
fmanKgTestInputFqId = FMAN_KG_DEMO_FQID;
|
|
|
|
/* Load our key generator scheme. */
|
|
|
|
fmanKgSchemeCreate (pDev, fqid, 1, FMAN_KG_DEMO_PORT);
|
|
|
|
/* Go into a busy loop, processing dequeue events. */
|
|
|
|
processLoop ();
|
|
|
|
/*
|
|
* We broke out of the loop, delete the scheme we created
|
|
* and clean up before exiting.
|
|
*/
|
|
|
|
fmanKgSchemeDestroy (pDev, fqid, 1, FMAN_KG_DEMO_PORT);
|
|
qportalFqShutdown (portal, fqid);
|
|
qportalIntHandlingEnable (portal);
|
|
qportalRelease (cpu, portal);
|
|
|
|
err:
|
|
|
|
return;
|
|
}
|
|
|
|
/*****************************************************************************
|
|
*
|
|
* fmanKgSmpDemoStop - stop the key generator demo
|
|
*
|
|
* This function terminates the demo started by fmanKgSmpDemoStart(). The
|
|
* key generator scheme is removed, the frame queue previously created is
|
|
* shut down, queue manager event handling is re-enabled and the polling task
|
|
* is terminated.
|
|
*
|
|
* RETURNS: N/A
|
|
*
|
|
* ERRNO: N/A
|
|
*/
|
|
|
|
void fmanKgSmpDemoStop
|
|
(
|
|
void
|
|
)
|
|
{
|
|
|
|
fmanKgTestQuit = TRUE;
|
|
return;
|
|
}
|