Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 472

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 112

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/jerdew5/orbworks.com/forum/includes/bbcode.php on line 112
[phpBB Debug] PHP Notice: in file /includes/functions.php on line 3368: Cannot modify header information - headers already sent by (output started at /includes/bbcode.php:472)
[phpBB Debug] PHP Notice: in file /includes/functions.php on line 3370: Cannot modify header information - headers already sent by (output started at /includes/bbcode.php:472)
[phpBB Debug] PHP Notice: in file /includes/functions.php on line 3371: Cannot modify header information - headers already sent by (output started at /includes/bbcode.php:472)
[phpBB Debug] PHP Notice: in file /includes/functions.php on line 3372: Cannot modify header information - headers already sent by (output started at /includes/bbcode.php:472)
OrbWorks Community Forum • View topic - Can anyone help me speed up this code???

Can anyone help me speed up this code???

Discuss PocketC for CE (including Desktop Edition)

Postby Glasswalker on Thu Apr 17, 2003 3:35 am

I'm having problems with this code... as in it takes a week to run.
I can't seem to find anywhere to simplify it at all... so if anyone can help I would really appreciate the help...

strint(string str)
{
int offset,i,outval,outvalt,neg;
offset=0;
outval=0;
outvalt=0;
neg=0;
i=0;
while(i<strlen(str))
{
if(i==0&&strgetc(str,i)!='0'&&strgetc(str,i)!='-')
break;
if(offset==0)
if(strgetc(str,i)!='0'&&strgetc(str,i)!='-')
offset=i;
if(strgetc(str,i)=='-')
neg=1;
i++;
}
for(i=offset;i<strlen(str);i++)
{
outvalt=pow(10,(strlen(str)-i)-1);
if(outvalt==0)
outvalt=1;
if(strgetc(str,i)=='0')
outval=outval+(outvalt*0);
if(strgetc(str,i)=='1')
outval=outval+(outvalt*1);
if(strgetc(str,i)=='2')
outval=outval+(outvalt*2);
if(strgetc(str,i)=='3')
outval=outval+(outvalt*3);
if(strgetc(str,i)=='4')
outval=outval+(outvalt*4);
if(strgetc(str,i)=='5')
outval=outval+(outvalt*5);
if(strgetc(str,i)=='6')
outval=outval+(outvalt*6);
if(strgetc(str,i)=='7')
outval=outval+(outvalt*7);
if(strgetc(str,i)=='8')
outval=outval+(outvalt*8);
if(strgetc(str,i)=='9')
outval=outval+(outvalt*9);
}
if(neg==1)
outval=outval*-1;
return outval;
}

LoadData()
{
string tempchar,columndata[6];
int mwdahandle,columncount,firstline;
string mwdafiledata;
firstline = 0;
recordcount = 0;
for(i=0;i<6;i++)
wndshow(501+i,0);
DrawNewFrame(4,4,14,3);
DrawNewText("Please Wait... Loading Database...",5,5);
mwdahandle = fileopen("\\Program Files\\Mechwarrior\\Mechwarrior.csv",0,0);
if(mwdahandle==-1)
alert("Database Open Error!");
mwdafiledata = fileread(mwdahandle,65535);
fileclose(mwdahandle);
for(i=0;i<strlen(mwdafiledata);i++)
{
tempchar = strmid(mwdafiledata,i,1);
if(strncmp(tempchar,",",1)==0)
{
columncount++;
}
if(strncmp(tempchar,"\r",1)==0)
{
columncount = 0;
if(strncmp(columndata[0],"This",4)==0)
break;
for(j=0;j<6;j++)
{
if(firstline>=6)
{
if(j==0)
db[recordcount].number = columndata[j];
if(j==1)
db[recordcount].count = columndata[j];
if(j==2)
db[recordcount].name = columndata[j];
if(j==3)
db[recordcount].rank = columndata[j];
if(j==4)
db[recordcount].faction = columndata[j];
if(j==5)
db[recordcount].points = columndata[j];
}
columndata[j] = "";
firstline++;
}
if(firstline>6)
recordcount++;
}
if((strncmp(tempchar,",",1)!=0)&&(strncmp(tempchar,"\r",1)!=0)&&(strncmp(tempchar,"\n",1)!=0))
columndata[columncount] = columndata[columncount] + tempchar;
}
for(i=0;i<6;i++)
wndshow(501+i,5);
}
Glasswalker
 
Posts: 47
Joined: Wed Mar 21, 2001 12:18 am
Location: Canada

Postby KOU on Thu Apr 17, 2003 4:37 am

A quick look, so there might be more

lots of
if(strgetc(str,i)=='x')

to

char cTemp;
cTemp = strgetc(str, i);

if('0' <= cTemp && cTemp <= '9')
cTemp = cTemp - '0'; // might be different. may be +1 or -1

outval=outval+(outvalt*(int)cTemp);

I think many parts of my code is wrong.
And I'm not sure that this is faster.
You'll need to debug the code first.
Completely untested. Sorry...

-----------------------------------------

strint function...
Is it converting from string to int?
then can't you just cast'em
Ex. [int var] = (int) [string var]

If you want to use your own...
First, change pow to ^ operator.
Pow is ^ for floats. It is slow.

You might want to change the algorithm

instead of
base = 10^basecount;
Loop:
ret = ret + (int)[new char] * base;
get [new char] and loop
GOTO Loop;

try
Loop:
ret = 10*ret + (int)[new char];
GOTO Loop;

----------------------------------------

outval=outval+(outvalt*0);

any thing times 0 is 0!!!!

So comment it out.

Sorry for the long post.

KOU

Editted out mistakes
KOU
 
Posts: 92
Joined: Mon Mar 18, 2002 8:04 pm
Location: USA

Postby Glasswalker on Thu Apr 17, 2003 6:52 pm

Thanks a ton for the help with the strint() function... I for some reason had no clue that you could directly cast from a string to an int... That is really quite handy...

Well I have shortened the hell out of that...

But now the LoadData function needs some DESPERATE cleaning up... it is the one that takes the longest

(when I start my program it takes like 4 minutes to load the datafile... and the file is only 15K big... however the save routine only takes a matter of seconds)

I really need a way to speed it up...

I am parsing a comma seperated file there, so is there another fast way to parse a CSV file other than one character at a time?

Thanks
Glasswalker
 
Posts: 47
Joined: Wed Mar 21, 2001 12:18 am
Location: Canada

Postby wolfgang on Fri Apr 18, 2003 6:12 am

Make a search on this webboard. Some time ago Guy Sprackland posted complete & tested code to read and parse text files (I think he even handled CSV format).

Wolfgang


Composing music on the PocketPC! - http://www.pdamusician.com
Composing music on the PocketPC! - http://www.pdamusician.com
wolfgang
 
Posts: 281
Joined: Tue Dec 12, 2000 4:22 am
Location: USA

Postby Mickle on Wed Apr 23, 2003 5:51 am

Sorry for my English.
.
In LoadData() you use db[recordcount].number..... So. It mean that you use global structs.
PocketC V3 cannot work with global structs - DO NOT USE it. (links &MyStruct in your code not will work).
.
Use CEAPI library and real memory. It will speed up your program.
If your file are big (> 20 KB) you will have UNACCEPTABLE slow implementation with PocketC strings and/or PocketC arrays.
If you not want using real memory then use PocketC V2 instead PocketC V3. PocketC V2 faster up to 10 times (most common - 2 times).
But PocketC V3 + CEAPI library can be faster up to 50 - 100 times (obvious depends on what you do, when use C RTL....).
.
You need in strint() if you want to extract ints from any position in strings.
.
I extract code from very old files. I not remember was tested these functions or not.
But I compile it... - will work.
GetData() is only example how to do it faster. I think in real program linked list + index will better than array.
Speed - 4.5 KB per second (9 KB/sec UNICODE). Probably it best what you can do on PocketC V3 without CEAPI.
Maybe you can improve performance but not greater than 5 - 10 percents.
Code: Select all
strint(string s, pointer pindex)
{
   int i = 0, c, result = 0, sign = 1;

   if (pindex) i = *pindex;
   if (strgetc(s, i) == '-')
   {
      sign = -1; i++;
   }
   for (; c = strgetc(s, i); i++)  // strgetc() return '0' when i >= strlen(s, i).
   {
      if (c >= '0' && c <= '9') result = result * 10 + c - '0'; else break;
   }
   if (pindex) *pindex= i;
   return (result * sign);
}

NumberOfColumns(string data, char delim)
{
   char c; int i, result = 1;

   for (i = 0; c = strgetc(data, i); i++)
   {
      if (c == delim) result++; else if (c == '\n') break;
   }
   return result;
}

// ~9.5 KB per second (ASCII).

NumberOfRecords(string data)
{
   char c; int i, result = 0;

   for (i = 0; c = strgetc(data, i); i++) if (c == '\n') result++;
   return result;
}

// ~4.0 - 5.0 KB per second (ASCII).

GetData(string data, pointer records, int num_of_columns, int max_records, char delim)
{
   char    c;
   int     i = 0, column = 1, num_of_records = 0, length = 0, index = 0;

// To allow use *records++ = s; instead records[record][column] = s;
   records = records + max_records;
   while (c = strgetc(data, i++))
   {
      if (c == delim)
      {
          if (column < num_of_columns)
          {
             *records++ = strmid(data, index, length);
             length = 0;
             index  = i;
             column++;
          }
          else length++;
      }
      else if (c == '\n')
      {
         *records++ = strmid(data, index, length);
         if (column < num_of_columns) records = records + num_of_columns - column;
         length = 0;
         index  = i;
         column = 1;
         if (++num_of_records == max_records) break;
      }
      else if (c != '\r') length++;
   }
   return num_of_records;
}

// Creates double dimension array. Usual prototype must be &YourStruct.

SctArray(int size, int num, pointer prototype)
{
   pointer q, p;

   if (p = malloc(num + num * size))
   {
      p = p + num;
      q = p + num * size;
      if (prototype)
      {
// memcpy() copy and source types to destination.
// DO NOT USE settype() when you needed set type to 's' and array are big (> 1000)
// It will very slow down your program. Use memcpy() instead.
// If size of array of strings > 10000 and you use settype(array, 10000, 's') then your program will unusable.
// Sorry for it loop but PocketC V3 SO slow...
         while (num--) memcpy(*--p = (q = q - size), prototype, size);
      }
      else
      {
         while (num--) *--p = (q = q - size);
      }
   }
   return (p);
}

// ~1 MB per second. mode == 0 - ASCII file, mode !=0 - UNICODE file

LoadFile(string path, int mode)
{
   int file; string data;

   if ((file = fileopen(path, mode, 0)) != -1)
   {
      if (mode) fileseek(file, 2, 0);
      data = fileread(file, filegetlen(file));
      fileclose(file);
   }
   return (data);
}

#define MAX_RECORDS    1000
#define NUM_OF_COLUMNS 52  // number of columns in my test file.
#define DELIMITER '\t'     // ',' when you use CSV

struct MyRecord {
   string s[NUM_OF_COLUMNS];
};

main()
{
   string  data;
   pointer records;
   int     time, i, j, num_of_columns, num_of_records;
   struct MyRecord prototype;

   data = LoadFile(OpenFileDlg("Text Files|*.txt|*.csv"), 0);
   num_of_columns = NumberOfColumns(data, DELIMITER);
   records = SctArray(num_of_columns, MAX_RECORDS, &prototype);
time = ticks();
   num_of_records = GetData(data, records, num_of_columns, MAX_RECORDS, DELIMITER);
time = ticks() - time;
alert("Time: " + time);
   putsl("Columns: " + num_of_columns);
   putsl("Records: " + num_of_records);
   for (i = 0; i < 10; i++)
   {
      for (j = 0; j < num_of_columns; j++)
      {
         puts(records[i][j] + "\t");
      }
      putsl("");
   }
   showconsole();
}
Mickle
 
Posts: 48
Joined: Wed Apr 17, 2002 1:22 am
Location: Russia

Postby Mickle on Thu Apr 24, 2003 12:08 am

GetData() - contains bug (when MAX_RECORDS < actual number of records...).
In addition it cannot work when file contains line(s) with number of columns < than in first line (or > than in first line).
Improved version will work in these cases.
Code in post above are updated (GetData() and main()).
Mickle
 
Posts: 48
Joined: Wed Apr 17, 2002 1:22 am
Location: Russia


Return to PocketC for CE

Who is online

Users browsing this forum: No registered users and 1 guest

cron