APRIL 1992 VOLUME 4 NUMBER 4

INDEX

• MAD'S COLUMN
• ARTICLES
• TECHNICAL INSIGHTS
• COMING EVENTS: 1992 APPLE DEVELOPER CONFERENCE
• NIBBLES AND BITS
• NOVELL EDUCATION
• FUN & FACTS
• CONTACTING NOVELL
• ACKNOWLEDGEMENTS

MAD'S COLUMN

Last month, I found it interesting that within one week two trade magazines stated that NLM development is difficult. A little later, a Novell VP asked me if writing an NLM was hard. Two questions popped into my head. "How do you define 'hard?'" and "Is this a trick question?" I mean, NetWare is the best platform for running high-performance applications, a good compiler and excellent documentation are available, and Novell offers specialized training for NLM development.

I posed the question to several folks in Developer Support with varying levels of NLM experience. During that same week, our Education Department was presenting the "Developing NLMs" class on site. At the end of the course, I asked the students (still new to NLM development) to comment on the statement:

"Writing an NLM is easy!"

The answers were not as varied as you might think. There were two predominant answers:

1. Writing a simple NLM is easy.
2. Writing all multi-threaded applications takes a certain mindset, but it is comparable to writing a Windows or OS/2 multi-threaded program.

• Resolve the compatibility issues between NetWare versions and API sets.

Happy_Programming!

Mad Poarch
Director
Developer Support/Service

ARTICLES:

NLMs: From "Hello World" to World-Class Computing (Part I)

Comparing Environments

  #include <stdio.h>

  main()
  {
    printf("Hello World");
  }

Next is a version of "Hello World" that will run as an NLM.

  #include <stdio.h>

  main()
  {
    printf("Hello World");
  }

Compiling an NLM is very straight-forward; in fact, the process is identical to compiling applications for DOS, OS/2 and Windows. The WATCOM C/386 compile line for the "Hello World" example for NLMs and an explanation of each parameter is displayed below:

   wcc386 /3s /Zp1 /d2 hello.c

To produce an executable load module, you would issue the command:

   wlink @hello.lnk

Since WATCOM C/386 also supports Extended-DOS and OS/2 v2.0, and both environments use the flat memory model, you could link HELLO.OBJ into an NLM, a protected-mode DOS application, or an OS/2 v2.0 application! The WATCOM Linker, WLINK, is the tool that creates the load module for each environment. For the "Hello World" NLM, you would use the linker definition file displayed in Figure 1. The end result of this process is an NLM that can be loaded and run on the server with the command, "load hello."

FIGURE 1: Linker definition file for "Hello World"

   form    novell nlm 'Hello World'
   name    hello
   option  caseexact, nod, stack=8k, map
   debug all debug novell
   file prelude, test
   module clib
   option screenname 'Hello World Screen'
   option threadname 'Hello World '
   import @clib.imp

End of Figure 1

Adding Functionality

• PressAnyKeyToContinue()
  Displays message and waits for a key
• HideInputCursor()
  Turns off the cursor
• DisplayInputCursor()
  Turns on the cursor

FIGURE 2: Required changes for BINDSCAN

#if !defined(__386__)
  #define PressAnyKeyToContinue()    \
{ \
 printf("\nPress any key to continue\r"); \
 getch();   \
}
  #define HideInputCursor()
  #define DisplayInputCursor()
#endif

End of Figure 2

   wcl /zp1 /d2 bindscan.c snit.lib

In the discussion on building the DOS version of BINDSCAN, the example linked in a static library called SNIT.LIB. This library contains all of the NetWare APIs for the DOS environment. In fact, the DOS version also requires that you link it with another static library, CLIBS.LIB. This library contains APIs for WATCOM's implementation of the standard C library.

In the NLM environment, a dynamic library called CLIB.NLM is provided with NetWare 3.x. This library provides most of the APIs available to NLM developers. (See sidebar: The NetWare C Runtime Library NLM.) This is not to say that statically-linked libraries can not be used in the NLM environment, because they can. A utility called WLIB, an object librarian included with the Network C for NLMs SDK, allows you to create static library files which can be linked in with WLINK.

Sidebar: Novell's C Runtime Library NLM

API                 DOS  NLM  OS2

printf              X    X    X
open                X    X    X
fopen               X    X    X
signal                   X    X    X
exit                X    X    X
sprintf                  X    X    X
getch               X    X    X
strcmp              X    X    X
malloc                   X    X    X
BeginThread         na   X    1
SuspendThread  na   X    2
EnterCritSec        na   X    3
CreateBinderyObject X    X    4
ScanProperty        X    X    5

1    Function called '_beginthread' in OS/2 C Library.

2    Function called 'DosSuspendThread' in OS/2 C Library.

3    Function called 'DosEnterCritSec' in OS/2 C Library.

4    Function called 'NWCreateBinderyObject'

5    Function called 'NWScanProperty'

End of Sidebar

multi-threaded Applications

Both OS/2 and NetWare provide the capability to create multi-threaded applications. Developing multi-threaded applications for either of these environments poses similar challenges. Some differences between these platforms surface when programming for factors like preemption. Preemption requires you to lock down data structures, since a thread may be suspended at any time. If the update of one or more data structures must be performed as a canonical operation, you must provide synchronization to prevent corruption of the data. For example, if a linked list of data structures is shared by all threads of execution and you need to remove a link from the list, you must "lock" the list while removing the link to prevent another thread from trying to traverse it during this process.

Even in a non-preemptive environment such as NetWare 3.x, there will be situations where multiple threads compete for some type of resource requiring serialized access. If one thread is accessing such a resource and gives up control (perhaps implicitly), another thread may begin to use it which may cause problems. In situations like this one, you must synchronize threads.

There are many ways to synchronize threads, but the easiest is by using local, in-memory semaphores. Both OS/2 and NetWare provide this type of mechanism, with the main difference being only in the API names. Another option for thread synchronization is accomplished by disabling CPU interrupts. If the work you are doing is very short, like removing a link from a linked list, disabling interrupts is usually easier, and carries less overhead. Of course, disabling interrupts is not always an option. If the operating system supports Intel processor ring protection, applications that are not running at the IOPL privilege level are not allowed to disable interrupts.

To illustrate a method for performing thread synchronization, an example program named DBLWIN.C has been included with this article's source listings (see "DAX (Distributed Application Example") for instructions on obtaining the source). DBLWIN.C illustrates the use of local semaphores to serialize access to the local screen's current cursor position. In addition, it creates a debug screen for debug messages, and synchronization to the current screen is also performed using local semaphores. The NLM divides the main screen into two logical windows, then creates two threads, each one being responsible for keeping the time accurate in its assigned window.

Two Multi-thread Design Strategies

In the "dynamic worker thread" strategy, each time a request is received by the server, a thread is started to service the request. This approach is easy to implement, as it requires that you simply spawn a new thread for each new request. On the other hand, this method can consume large amounts of resources and create sufficient overhead on the kernel's run queue. Just considering stack space requirements, a normal thread usually takes about 8K of RAM for its local stack. If you wanted to support 250 clients simultaneously, you would consume two megabytes of RAM just for the thread stack! You must also consider that 250 threads competing for the CPU would increase the amount of time that all threads must wait before regaining CPU control.

In the alternate "pool of threads" approach, a number of worker threads are started when the server application begins, then, as work comes in, a thread is taken from the pool to service it. This prevents the situations listed above from occurring, however it adds the additional overhead of queuing requests in situations where all threads in the pool are already busy.

Which strategy is best for your application? Maybe the first, maybe the second, and maybe a combination of both. It really depends on your application, and you must apply the characteristics of your program to the designs, to determine the best strategy for you.

DAX (Distributed Application Example)

The software portion of any distributed application can be broken down into four major components:

• Communication protocol
• Distributed application protocol (DAP)
• Client application
• Server application

"In the real world, you see client-server relationships in almost every aspect of life. More often, people play the role of client. For instance, when Charlie wants to buy something from a mail-order house, Charlie is the client; the mail-order house is the server; the order form that Charlie fills out is the client-server protocol, speaking in a request/reply language which the order form provides.

"The mail service used to send and receive the order form is the communication protocol (this includes the type of envelope, the addressing scheme, and the method of delivery). The carrier (US Mail, Federal Express, or whatever) is the communication media."

In order to isolate the communication protocol (e.g., IPX or TCP/IP) from the DAP, the example application contains a layer of software and a generic API that separates one from the other. This allows you to change the communication protocol (CP) layer, without changing any of the layers above, including the DAP as well as the client and server applications.

The current version of the example program supports both DOS and NLM clients. To isolate the CP layer from those above, the example application uses a static library containing object modules for both DOS and NLM implementations. The communication protocol used in this example is IPX. The source code can be compiled for either supported environment by defining one of two constants:

"DOSCLIENT"
or
"NLMCLIENT"

On the server side, the CP layer is designed as a standalone NLM which exports several APIs to provide its services to the other application (the server part of the distributed application). The CP layer could have been implemented as a static library, but if you were to add support for other protocols, a static library would force you to rebuild the entire server application, rather than just the underlying communication protocol layer NLM.

DAX is very modular in nature and uses identical file names for the client and server modules to make it easy to match up the pieces. The CP layer is broken into two logical pieces, the client side and the server side. The two sides share a common header file, CP.H, which defines the layout of the data exchanged between the two nodes on the network. The type declaration below displays the structure of the packet defined in CP.H.

typedef struct {

   CPHEADER   cphdr;

   UINT8 msg[CPMAXMSG];

}CPMESSAGE;

The "cphdr" member contains the information that the CP layer uses to maintain communications between the nodes. In DAX, "cphdr" consists of a signature (for authentication purposes), a server ID for identifying the client node to the server application, and some reserved bytes which could be used for implementing things like large packet support or flow control on connectionless-oriented protocols. Figure 3 contains the structure of the packet header defined in the CP.H structure.

FIGURE 3: Structure of the packet header defined in CP.H

typedef struct {
   UINT32  signature; // signature for packet CPPKTSIG
   UINT32  serverID;  // server ID for this session
   UINT8   reserved[6]; // keep six bytes for later
}CPHEADER;

End of Figure 3

Software Modules

• CPINIT
• CPCONN
• CPSEND
• CPRECV

CPINIT Module

CPCONN Module

On the server side, this module contains the routines that accept new sessions with the application server and maintain the array containing client information. Because this version of DAX is connectionless, it accepts new connections dynamically as they are received. On receipt of a client request, DAX uses the sessionID from the CP header to index the client data array. If the network address of the new request does not match the address in that element, a new slot is assigned for the client.

CPSEND Module

CPRECV Module

On the server, this module handles incoming requests from client applications. This module is perhaps the most complex of all server-side CP modules. Like the other modules, it has both initialization and de-initialization routines. It also has an independent thread of execution which handles incoming requests. As new requests are received, this thread identifies the requester, and then calls an API in the DAP layer to process the request. To avoid overloading the receiving thread, the API in the DAP layer queues the request for later processing. This design allows the server application to repost the ECB to listen for new requests.

In May, Bullets will examine the DAP layer and the client and server applications that use it. Next month's issue will also discuss how to port the example application so that you can create your own distributed application for NetWare 3.x. So, stay tuned next month for Part Two of "NLMs: from 'Hello World' to World-Class Computing."

TECHNICAL INSIGHTS:

Btrieve DATE Data Type & Microsoft BASIC (Btrieve (All versions))

To efficiently sort on a date field, create a structure using the user-defined TYPE and END TYPE statements for the key buffer.

   TYPE Date
  Day   AS STRING * 1
  Month AS STRING * 1
  Year  AS INTEGER
   END TYPE

   DIM Birthdate AS Date

   Global Birthdate AS Date

   Birthdate.Day   = MKI$(20)
   Birthdate.Month = MKI$(6)
   Birthdate.Year  = 1999

   Birthdate.Day   = CHR$(20)
   Birthdate.Month = CHR$(6)
   Birthdate.Year  = 1999

Once these values have been inserted into the Btrieve file, the same birthdate structure can be used to retrieve data with a GET operation. Once the record is returned into the birthdate structure, convert the day and month variables back to their numeric values.

(For BASIC PDS v7.x)

CALL BTRVFAR(GETFIRST%,
   PosBlk$,
   VARPTR(Birthdate),
   VARSEG(Birthdate),
   _BufLen%,
   KeyBuf$,
   KeyNum%)

(For Visual Basic v1.0)

Status%=BTRCALL(GETFIRST%,
 PosBlk$,
 Birthdate,
 BufLen%,
 KeyBuf$,
 KeyBufLen%,
 KeyNum%)

Day%   = ASC(Birthdate.Day)
Month% = ASC(Birthdate.Month)
Year%  = Birthdate.Year

Preventing NLMs from Unloading

Insert the following API call before the "return" statement at the end of the routine:

  DestroyScreen(NewScrID);

Specifying Server Name/Volume Name in BLOG.CFG (NetWare Btrieve (NLM) v5.15)

To ensure that BROLLFWD functions properly, use a drive letter to specify the path to the log file.

maxECBs in Windows IPXInitialize() (NetWare C Interface for Windows v1.22)

In this mode, passing zero for maxECBs is allowed only for compatibility with the Standard mode The zero setting for maxECBs will be ignored and VIPX will manage ECB allocation, so applications should allow VIPX to perform this task. However, in Standard mode, applications can pass zero for maxECBs and then manage their own ECB allocation.

GetBroadcastMessage API Update (NetWare C Interface for DOS v1.2)

FIGURE 4: Necessary modifications to BRODCAST.C

(Changes marked with "<---")

int GetBroadcastMessage(messageBuffer)
char *messageBuffer;
{
    char sendBuf[3], replyBuf[61]; <---
    int len, ccode;

    *((int *)sendBuf) = 1;
    sendBuf[2] = (char) 1;
    *((int *)replyBuf) = 59;  <---
    replyBuf[2] = (char)58;   <---
    ccode = _ShellRequest ((BYTE)0xE1, (BYTE *)sendBuf,
 (BYTE *)replyBuf);
  if (ccode)
    return (ccode);
    len = (int)replyBuf[2];
    memmove(messageBuffer, replyBuf + 3, len);
    messageBuffer[len] = 0;
    return (0);
}
End of Figure 4

Brequest Hangs If No Network Connection (NetWare Btrieve (NLM) v5.15)

SET B_LOAD=Yes
IPX
NETX
IF ERRORLEVEL 1 SET B_LOAD=NO
...
IF %B_LOAD% == Yes BREQUEST /D:8192
SET B_LOAD=
...

xRestrict & Views with Restrictions (XQL (all versions))

  CREATE VIEW REST AS

    SELECT * FROM PATIENTS

    WHERE ZIP = "78759"

  SELECT * FROM REST

    WHERE Last^Name BEGINS WITH "A"

Using XQL Primitives, an application would perform an xRecall of the VIEW "REST" and then an xRestrict ("Last^Name BEGINS WITH "A"). However, the overall restriction on the VIEW will be determined by which iOption is specified on the xRestrict call .

If you set the iOption parameter specified in the xRestrict call to one, the new restriction will replace the previously defined restriction. If you set iOption to zero, the new restriction will be appended to the previous restriction with an "AND" operator. You can also set iOption to zero and precede the restriction statement with the "OR" operator (|| Last^Name BEGINS WITH "A"). Then, XQL will append the new restriction to the previous restriction with an "OR" operator.

Loading DLLs Explicitly in Windows (NetWare C Interface for Windows v1.22)

To avoid this situation, load the DLLs explicitly instead of implicitly. To load the DLLs, follow the steps outlined in Figure 5.

FIGURE 5: Steps for loading DLLs explicitly in Windows v3.0

A)   Determine if the NETWARE.DRV is loaded:

   HANDLE hNetWare;

     hNetWare = GetModuleHandle("NETWARE.DRV");
     if (hNetWare == NULL)

// not loaded
        else

// loaded!

B)   Try to load the DLL (in this example, NWBIND.DLL):

   HANDLE hDLL;

hDLL = LoadLibrary("NWBIND.DLL");

if (hDLL < 32)

// error returned (file not found = 2)

   else

// loaded!

C)   Make functions point to their location(s) in the DLL:

// This method uses the function string name

// (very memory hungry; pointers + strings, slow load time)

   GetBinderyObjectID =

(WORD (FAR PASCAL *)()) GetProcAddress(hDLL,

"GetBinderyObjectID");

   // This method uses function ordinal number

 (in this example, 14)

   // (little memory; just pointers, very fast)

   GetBinderyObjectID =

   (WORD (FAR PASCAL *)()) GetProcAddress(hDLL, (LPSTR)14);

End of Figure 5

FIGURE 6: Modification to GetBinderyObject() required when loading DLLs explicitly

Old:

extern WORD FAR PASCAL

   GetBinderyObjectID(char   far *objectName,

  WORD   objectType,

  DWORD  far *objectID );

New:

WORD (FAR PASCAL

  *GetBinderyObjectID)(char   far *objectName,

   WORD   objectType,

   DWORD  far *objectID );

End of Figure 6

Relocation of Source Code for Bullets Articles

COMING EVENTS: 1992 APPLE DEVELOPER CONFERENCE

This also will be your first opportunity to see DataClub from Novell, the newest addition to Novell solutions for the Macintosh. With DataClub, Macintosh users enjoy the high performance of a dedicated file server with or without dedicated server hardware.

Mark your calendar for May 13 and plan on visiting the Enterprise Systems Expo, at the 1992 Apple Developer Conference.

NIBBLES AND BITS

Fast Answers to Common Questions

Btrieve

A - Status 1015 is returned by Btrieve for OS/2 if you have specified the following environment variable before starting your application:

     SET BTRPARMSCHK=Y

Q - Can I load the Btrieve/NetWare SQL NLMs manually from the console? What will happen if I use the NCF files to load them?

A - If you load these NLMs with the manual 'Load <NLM name>' command at your server console or in your AUTOEXEC.NCF file you may experience the following problems:

• If you fail to load the corresponding SPX communication NLM (BSPXCOM, or NSSPXCOM), you will receive errors when calling Btrieve or NetWare SQL from a workstation, another server, or when loading BCONSOLE.NLM.
• If you manually load either BTRIEVE.NLM or NWSQL.NLM, you receive nonzero status codes when you exceed the default load parameters for the respective NLM. The load parameters that you set in BSETUP or NSSETUP are written to BSTART.NCF or NSSTART.NCF respectively, so if you do not load using the NCF file, you are loading the NLMs with default parameter values. You can also call the NCF files from the server's AUTOEXEC.NCF file to load the NLMs automatically with the correct parameters.

NetWare for SAA v1.2

A - No, NetWare for SAA v1.2 supports both 5250 and 3270 mainframe and miniframe connections.

Xtrieve PLUS for DOS v4.10

     field1 field2 field3

NetWare C Interface for DOS v1.2

A - The encryption algorithm is part of the login source code, and is not available to the public. You can, however, obtain a "generic" AsmLoginToFileServer() and related functions that can be called by any compiler that supports both a code and data segment and C-style parameter passing on the stack. Currently, there is no support for Turbo Pascal-based calls to these functions, as Turbo Pascal requires there to be no data segment. Contact the Developer Support Group at Novell Austin to request this set of object files.

VIPX.386 v1.10

Network C for NLMs v2.0b

1. Drop into the Internal Debugger.
2. Set a breakpoint exactly like this:

        b=malloc-4d+[dmalloc-9]
   

3. Run your NLM.
4. When the NLM exits, the debugger is entered once for each pointer. In the debugger, register EAX contains a pointer to un-freed memory.

OS/2 Developer's Kit v1.3a

NOVELL EDUCATION

• 901 - Programming with NetWare Client APIs
  June 10-12 Sunnyvale, CA
• 905 - Programming with Btrieve
  May 12-14 Austin, TX
  July 8-10 Sunnyvale, CA
• 906 - Programming with Btrieve: Advanced Features
  June 16-18 Austin, TX
• 907 - Xtrieve PLUS
  May 20-21 Sunnyvale, CA
  July 14-15 Dallas, TX
• 910 - Programming with XQL
  June 10-12 Austin, TX
• 920 - Programming with Network Communication Services
  June 15-16 Sunnyvale, CA
• 930 - Developing NetWare Loadable Modules (NLMs)
  May 13-15 Sunnyvale, CA
  July 15-17 Washington, DC

For information on availability, location, and prices of international classes, contact your local Novell office.

FUN & FACTS

1.   How many bits are in an "int" in NLM environment?

A.   8

B    16

C.   24

D.   32
2.   What CLIB.NLM function generates unique strings for use as a valid filename?

A.   UniqueFileName()

B.   tmpnam()

C.   tmpfile()
3.   In the NetWare C Interface for DOS v1.2, what is the name of the header file that contains
the definitions for all the C structures needed for IPX and SPX?

A.   nxt.h

B.   ipxspx.h

C.   nit.h
4.   In the NetWare C Interface for DOS v1.2, what is the name of the function that initializes
the ECB's immediate address field?

A.   IPXGetLocalTarget()

B.   IPXImmediate()

C.   IPXInitialize()
5.   What are the two modes of services provided by TLI (Transport Layer Interface)?

A.   Connection mode & Connectionless mode

B.   Real mode & Standard mode

C.   Connect mode & Disconnect mode
6.   In NetWare SQL, which of the following is not a valid scalar function for string
manipulation?

A.   LENGTH

B.   REVERSE

C.   RIGHT

D.   UPPER
7.   If you use both XQLP and XQLM function calls, which Login function should you use?

A.   xLogin

B.   XQLLogin
8.   What is the predefined function key in Xtrieve PLUS to accelerate command file replay?

A.   F6

B.   F8

C.   F2

D.   F1
9.   Xtrieve PLUS v4.10 has a username called PUBLIC. What is the password for PUBLIC?

A.   PUBLIC

B.   PUB

C.   There is no password

D.   PUBLIC is not a valid username
10.  In Btrieve, what is the name of the file that contains the list of all Btrieve files to be
logged?

A.   BLOG.BTR

B.   BLOG.CFG

C.   BTRLOG.CFG
11.  In the NetWare 2.x environment, the bindery is composed of two files. What are the
names of these two files?

A.   NET$BIND.SYS and NET$PROP.SYS

B.   NET$BIND.SYS and NET$OBJ.SYS

C.   NET$BIND.SYS and NET$BVAL.SYS
FUN & FACTS ANSWERS:
1.   D
2.   C
3.   A
4.   A
5.   A
6.   B
7.   B
8.   C
9.   C
10.  B
11.  C

CONTACTING NOVELL

Developer Support

Voice

Fax

Developer BBS

E-mail: CompuServe

E-mail: MHS

E-mail: Internet (SMTP)

NetWire on CompuServe

Novell Products and SDKs

DeveloperNet Labs

Novell Developer Relations

Voice

Fax

NetWire on CompuServe

E-mail: MHS or Internet (SMTP)

ACKNOWLEDGEMENTS

Contributing authors: Linda Anderson, Vitek Boruvka, Neda Eslami, David Harris, Sudz Khawaja, Ken Lowrie, Rudy McNeese, Clint McVey, Mike Shoemaker, John E. Stewart, Aslam Tejani, Maggie Walczynski, Chip Webb

Special thanks to the Developer Support, Development, Developer Relations, Product Marketing, and Marketing Communications staff who contributed time and valuable input.

(c) 1992 Novell, Inc. All rights reserved.

Novell, the N design, NetWare, Btrieve, XQL and LAN WorkPlace are registered trademarks, NetWare System Calls for DOS, NetWare Loadable Module (NLM), NetWare SQL, NetWare Btrieve, Xtrieve PLUS, NetWare C Interface for DOS, NetWare System Interface Technical Overview, NetWare RPC, NetWare RPC 386, NetWare LU6.2, Report Executive, and Professional Development Series (PDS) are trademarks, and NetWire, Direct Connect and Professional Developers' Program (PDP) are servicemarks of Novell, Inc. Apple, Macintosh and AppleTalk are registered trademarks of Apple Computer, Inc. CompuServe is registered trademark of CompuServe Corporation. Microsoft is a registered trademark, and Windows is a trademark of Microsoft, Inc. IBM and OS/2 are registered trademarks, and SAA is a trademark of International Business Machines Corporation. Sun Microsystems and NFS are registered trademarks of Sun Microsystems, Inc. WATCOM is a trademark of WATCOM Systems, Inc.