Information Hiding Methods used by Flexlm Targets An Explanation of the Flexlm Seed Hiding System.	Not Assigned
8-Oct-1999	by Nolan Blender
	Courtesy of Reverser's page of reverse engineering	slightly edited by tsehp
fra_00xx 98xxxx handle 1100 NA PC
	There is a crack, a crack in everything That's how the light gets in
Rating	( )Beginner (X)Intermediate ( )Advanced ( )Expert

Flexlm has undergone a gradual evolution.  Due to pressure from crackers

the programmers who have written this code have improved the quality of the

protection it provides.  At the current stage of product evolution, it is

no longer possible to simply extract the seeds from the lc_init call, even

if in reality it is called from the lc_new_job call.  This essay will 

explain

the new methods used by Flexlm to hide relevant information needed to build

licenses.

For a full understanding of of how lm_new_job() works, I suggest that you

read the essay by Dan, as it will provide you with a complete understanding

of how the hiding is used in flexlm licensed targets.  This essay provides

auxillary information that will clarify exactly how the random data in the

job structure is combined with the vendor code data.





The current version of Flexlm uses special techniques to hide the encryption

seeds required to generate licenses.  Earlier versions simply stored the

information in global data, then passed the information into lc_init().  No

attempt was made to disguise the information, so of course examining the

initialized global area would reveal the seeds and the vendor codes.  From

that point, it was a simple matter to extract vendorcode5 and xor it with

data[1] and data[2], which would reveal the original seeds.  Later versions

hid the information using lm_new.c.



The process for storing the encrypted data involves storing one byte chunks

of data in the global space, and then reassembling the data in the routine

pointed to by l_n36_buf.  There is a lot of code which does not get 

executed,

however the references to the symbols is enough to keep the filler globals 

from

being compiled out.   The vendorname and the vendor keys that are returned

are correct.  The encryption seeds are not set to the real encryption seeds,

or for that matter, the encryption seeds xored with vendor key5, but the

encryption seeds xored with a random value, and then salted with values from

the vendor name and the first two vendor keys. The other task that this 

program

performs is to set l_n36_buff (note: two 'f's) to the decrypting function.

The routine takes two arguments, buf and v, where buf is a char * pointer 

that

will have the vendor name in it after the function is called.  v is a 

pointer

to a vendorcode structure that will be filled in, but the encryption seeds 

will

have been xored wih a random value.



The decrypting routine takes in 3 values; the job structure (a shared data

structure which is passed around between routines), the vendor_id, and

a pointer to the vendorcode structure, which contains the randomized seeds

and the correct vendor keys.



There are two modes of operation for the decrypting routine.  If a null

pointer for job is passed into the routine, the routine extracts the

correct values for the encryption seeds and stores them in the vendorcode

structure.  If a valid pointer is passed in for job, time based randomized

data is stored in the 12 bytes starting at v+8, then the returned seeds

xored with a select set of those 12 bytes.  That way the clear seeds are

not available until a routine that knows how to xor the correct 4 bytes

with the vendor keys is called.



A careful examination of the decryption routine in lm_new.c shows many

operations, and the seeds are xored with many values.  The main point

of interest from a seed extraction process is the selection of the

bytes from the job structure, since knowing that will enable us to get the

correct seeds from the structure.



        key->data[0] ^=

                (((((long)sig[0] << 0)|

                    ((long)sig[1] << 2) |

                    ((long)sig[2] << 3) |

                    ((long)sig[3] << 1))

                ^ ((long)(t->a[5]) << 0)

                ^ ((long)(t->a[0]) << 8)

                ^ x

                ^ ((long)(t->a[1]) << 16)

                ^ ((long)(t->a[4]) << 24)

                ^ key->keys[1]

                ^ key->keys[0]) & 0xffffffff) ;



sig, which is a value generated from the vendor_id, is shifted and ored with

encryption seed, as well as the randomized value x.  This really doesn't

matter, since this is accounted for already in the n_36_buf routine.  The

indicies used for the t->a[%d] values are what is of interest here.



The routine uniqcode in lmrand2.o was examined, and the the code which

generates the decrypting program was partly reversed.  It turns out that

the first character of the vendor name is extracted, then that value mod 20

is used to select the bytes used to decrypt the seeds.  Rather than reverse

the entire routine, a program to generate vendor codes was written based

on earlier reversing techniques, and an lm_new.c was generated for each of

the 20 different possibilities.  The values were then extracted, and a table

containing these values was generated.  I'm certain that the table could 

have

been extracted from the lmrand2.o file, but it was easier to get the data

from runs of lmrand2.  You can use Softice or DDE or adb to see what is

being done, the selection code happens right at the beginning of uniqcode.



Link this demonstration program with lm_new.obj, and run.  This program

has been tested on HPUX using the ANSI compiler, and under Windows NT

using Microsoft Visual C++ 5.0.



#include





/*

*

* Module: extr.c v1.0.0.0

*

* Description: Demonstration program to show interaction

*              between security code in lm_new.c and main

*              program.

*

* blendern

* 08-oct-1999

*

* Last modified: 08-oct-1999

*/



/*

* Decoding table type

*/



typedef struct decode_table_s {

        int offsets[4];

} decode_table_t;



/*

* really vendorcode5, but renamed to avoid conflict

* if Globetrotter headers are included.

*

* The encryption seeds are stored in the data part of this structure.

*

* the VENDOR_CODES are stored in the keys part of this structure.

*/



typedef struct my_vendorcode5 {

                            short type;    /* Type of structure */

                            unsigned long data[2]; /* 64-bit code */

                            unsigned long keys[4]; /*- [0]: Product features 

*/

                                                   /*- [1]: platforms */

                                                   /*- [2]: platforms */

                                                   /*- [3]: Expiration date/ 

*/

                                                   /*-      Key check */

                            short flexlm_version;

                            short flexlm_revision;

                            char flexlm_patch[2];

#define LM_MAX_BEH_VER 4

                            char behavior_ver[LM_MAX_BEH_VER + 1];

                          } MY_VENDORCODE5;



/*

* decryption table values.

*/



decode_table_t decode_table[] = {

        {3, 5, 4, 11}, /* index 0 */

        {9, 8, 3, 1}, /* index 1 */

        {8, 1, 2, 5}, /* index 2 */

        {2, 1, 10, 5}, /* index 3 */

        {3, 0, 1, 7}, /* index 4 */

        {1, 10, 3, 7}, /* index 5 */

        {7, 3, 5, 11}, /* index 6 */

        {0, 1, 9, 4}, /* index 7 */

        {0, 4, 1, 10}, /* index 8 */

        {11, 8, 1, 3}, /* index 9 */

        {8, 4, 2, 5}, /* index 10 */

        {6, 1, 0, 9}, /* index 11 */

        {4, 3, 8, 9}, /* index 12 */

        {0, 4, 2, 10}, /* index 13 */

        {3, 10, 8, 7}, /* index 14 */

        {1, 11, 0, 3}, /* index 15 */

        {6, 5, 1, 0}, /* index 16 */

        {0, 2, 4, 8}, /* index 17 */

        {5, 0, 1, 4}, /* index 18 */

        {10, 3, 5, 1} /* index 19 */

};





/* function that we get from lm_new.c */

extern int (*l_n36_buf)();



/* function pointer that will be set later. */

void (*l_n36_buff)();



/************************************************************

*

* extr: demostrate calls to lm_new

*

*/



void main(int argc, char *argv)

{

        int retcode;

        int i;

        int key_index;

        int tabindex1;

        int tabindex2;

        int tabindex3;

        int tabindex4;

        int decrypt_xor_value;



        MY_VENDORCODE5 teststruct;

        char myvendorname[1024];  /* This is the vendor_id */



        /* Job structure */

        struct s_tmp

        {

                int i;

                char *cp;

                unsigned char a[12];

        } myjob;



        /* Initialize job */

        myjob.i = 66;

        myjob.cp = 0;



        /*

         * ensure that l_n36_buff is null, since lm_new tests

         * for this before setting it.

         */

        l_n36_buff = 0;



        /* Get vendorname with clear keys but encrypted seeds */

        retcode = (*l_n36_buf)(myvendorname, &teststruct);

        if (retcode != 1)

        {

                printf ("Problem with call to lm_new initializer.\n");

                printf ("retcode = %d\n", retcode);

                return;

        }



        /* DEBUG see what was returned */

        printf ("After call to l_n36_buf routine.\n");

        printf ("myvendorname: %s\n", myvendorname);

        for (i = 0; i < 2; i++)

        {

                printf ("data[%d] = %08x\n", i, teststruct.data[i]);

        }

        for (i = 0; i < 4; i++)

        {

                printf ("keys[%d] = %08x\n", i, teststruct.keys[i]);

        }



        (*l_n36_buff)(&myjob, myvendorname, &teststruct);



        /*

         * extract the xoring value for the job now.

         *

         * the index into the decoding table is the first

         * character of the vendor name mod 20.

         */



        key_index = (int)(myvendorname[0] & 0xff) % 20;



        tabindex1 = decode_table[key_index].offsets[0];



        tabindex2 = decode_table[key_index].offsets[1];



        tabindex3 = decode_table[key_index].offsets[2];



        tabindex4 = decode_table[key_index].offsets[3];





        /*

         * use values to reverse xor which occurs if

         * valid job given.

         */

        decrypt_xor_value = ((long)(myjob.a[tabindex1]) << 0)

                | ((long)(myjob.a[tabindex2]) << 8)

                | ((long)(myjob.a[tabindex3]) << 16)

                | ((long)(myjob.a[tabindex4]) << 24);



        printf ("Decrypt xor value: %08x\n", decrypt_xor_value);

        printf ("seed1: %08x\n", teststruct.data[0] ^ decrypt_xor_value);

        printf ("seed2: %08x\n", teststruct.data[1] ^ decrypt_xor_value);



        return;

}



Here is one run of the program; you will get different results for

the decrypt xor value for each run, however the seeds will be the

same each time.







After call to l_n36_buf routine.

myvendorname: blenderd

data[0] = 6c116a9b



data[1] = adf8a253



keys[0] = c450f9f4



keys[1] = 4d12be88



keys[2] = f52bcf4d



keys[3] = 3309994c



Decrypt xor value: 37a2cfa5

seed1: ae37b151

seed2: 6fde7999





As Dan has mentioned in his essay, the important thing to observe is that

the decrypting function reveals the seeds which are encoded in the l_n36_buf

function.





The hiding methods that are used are quite good, but there are some 

weaknesses

that can be exploited.   Both seed1 and seed2 are xored with the same 

values,

so it may be possible to launch a brute force attack using 2**32 different

values.   If we are given seed1^secretval and seed2^secretval we can cycle

through the 2**32 values of secretval in search of the key.  It is generally

easier to simply dig the seeds out of the target.



An interesting point is that the vendor name isn't used at all in the

key generation process - if the license data and encyption seeds are left

unchanged, but the vendor name in the license and the vendor keys are 

changed,

the same license key results.  This agrees with Dan's findings.   I have 

tried

using quite different vendor keys, and the generated keys are the same, and

only depend on feature/hostid information and the encryption seeds.

Final 

Notes

This pretty much sums up the new protection.  Later versions

of FLEXlm never call the lm_new decoding routine with a null

pointer, so you never see plaintext seeds from this routine.

homepage links search_forms +ORC how to protect academy database
reality cracking how to search javascript wars
tools anonymity academy cocktails antismut CGI-scripts mail_reverser
Is reverse engineering legal?