Author Topic: Reducing Program Size In UNICODE And X64 Builds With Matt Pietrek’s LibCTiny.lib  (Read 11510 times)

0 Members and 1 Guest are viewing this topic.

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
     This is yet another continuation of my previous posts on this topic from here…

http://www.jose.it-berater.org/smfforum/index.php?topic=5085.0

…and here…

http://www.jose.it-berater.org/smfforum/index.php?topic=5000.0

However, we’ll start from absolute fundamentals.  From Jeffrey Richter and Christopher Nasarre’s book “Windows Via C\C++” published by Microsoft Press, which, by the way, covers Windows internals in much more detail than Charles Petzold’s “Programming Windows” book, they state this…

Quote
When you use Microsoft Visual Studio to create an application project the integrated environment sets up various linker switches so that the linker embeds the proper type of subsystem in the resulting executable.  This linker switch is /SUBSYSTEM:CONSOLE for CUI applications and /SUBSYSTEM:WINDOWS for GUI applications.  When the user runs an application, the operating system’s loader looks inside the executable image’s header and grabs this subsystem value.  If the value indicates a CUI-based application, the loader automatically ensures that a text console window is available for the application – such as when the application is started from a command prompt - and, if needed, another one is created - such as when the same CUI based application is started from Windows Explorer.  If the value indicates a GUI based application, the loader doesn’t create the console window and just loads the application.  Once the application starts running, the operating system doesn’t care what type of UI your application has.

Your Windows application must have an entry point function that is called when the application starts running.  As a C/C++ developer, there are two possible entry point functions you can use:

int WINAPI _tWinMain(HINSTANCE hInstanceExe, HINSTANCE, PTSTR pszCmdLine, int nCmdShow);
int _tmain(int argc, TCHAR* argv[], TCHAR* envp[]);

Notice that the exact symbol depends on whether you are using Unicode strings or not.  The operating system doesn't actually call the entry point function you write.  Instead,  it calls a C/C++ runtime startup function implemented by the C/C++ runtime and set at link time with the -entry: command line option.  This function initializes the C/C++ runtime library so that you can call functions such as malloc and free.  It also ensures that any global and static C++ objects that you have declared are constructed properly before your code executes.  Table 4-1 tells you which entry point to implement in your source code and when.

Code: [Select]
Table 4-1  Application Types and Corresponding Entry Points

Application Type                     Entry Point              Startup Function Embedded In Your Executable
=========================================================================================================
GUI Application that wants ANSI      _tWinMain (WinMain)      WinMainCRTStartup
characters and striungs

GUI Application that wants Unicode   _tWinMain (wWinMain)     wWinMainCRTStartup
characters and strings

CUI Application that wants ANSI      _tmain (main)            mainCRTStartup
characters and strings

CUI Application that wants Unicode   _tmain (wmain)           wmainCRTStartup
characters and strings

There’s quite a bit to say about the above, especially this…

Quote
The operating system doesn't actually call the entry point function you write.  Instead,  it calls a C/C++ runtime startup function implemented by the C/C++ runtime and set at link time with the -entry: command line option.  This function initializes the C/C++ runtime library so that you can call functions such as malloc and free.  It also ensures that any global and static C++ objects that you have declared are constructed properly before your code executes.

Let’s start with the latter…

Quote
It also ensures that any global and static C++ objects that you have declared are constructed properly before your code executes.

What they are talking about there are objects declared and instantiated at global scope outside of any function such as my GlobalClass object below…

Code: [Select]
// cl GlobalCon.cpp /O1 /Os
// App includes C Runtime.  126,464 bytes
#include <stdio.h>
FILE* fp=NULL;

class GlobalClass    // C++ Class definition at global scope, which is typical.  The
{                    // Class definition does not by itself, cause any instances
 public:             // of the class to be created, i.e., instantiated.
 GlobalClass()
 {
  fp=fopen("Output.txt","w");
  fprintf(fp, "Entering GlobalClass Constructor!\n");
  iCounter=1;
  fprintf(fp,"  iCounter = %d\n",iCounter);
  fprintf(fp,"Leaving GlobalClass Constructor!\n\n");
 }

 ~GlobalClass()
 {
  fprintf(fp, "Entering GlobalClass Destructor!\n");
  iCounter=0;
  fprintf(fp,"  iCounter = %d\n",iCounter);
  fprintf(fp,"Leaving GlobalClass Destructor!\n\n");
  fclose(fp);
 }

 private:
 int iCounter;
} gc;             // However, this line causes a GlobalClass object to be instantiated
                  // before the operating system calls the main() function.
int main()
{
 printf("Hello, World!\n");
 if(fp)
 {
    fprintf(fp,"Entering main()\n");
    fprintf(fp, "  Hello, World!\n");
    fprintf(fp,"Leaving main()\n\n");
 }
 getchar();

 return 0;
}

In the above code note this line at the trailing brace of the class definition…

Quote
} gc;             // However, this line causes a GlobalClass object to be instantiated
                  // before the operating system calls the main() function.

That will necessitate that the C/C++ runtime create an object of the class before calling the main function.  The code to do this, which is extremely, extremely involved, runs within the mainCRTStartup or WinMainCRTStartup referenced and referred to above.  Did I say it was complicated?  Matt Pietrek went into the details of this in his LibCTiny article I’ve also provided links for his topic …

“The Soft Underbelly Of Constructors”

Anyway, here is the output of the above program which I had to direct mostly to a text file to catch the creation and destruction output statements.  If you would try to output this to the console you would miss all the important stuff because it happens before and after the existence of the console…

Code: [Select]
Entering GlobalClass Constructor!
  iCounter = 1
Leaving GlobalClass Constructor!

Entering main()
  Hello, World!
Leaving main()

Entering GlobalClass Destructor!
  iCounter = 0
Leaving GlobalClass Destructor! 

Matt Pietrek provided the code in his LibCTiny.lib to take care of all this.  It adds maybe a kilobyte to the executable size.  But do we really need it?  I very seldom do.  In fact, I can only think of one instance where I’ve ever done this, and likely with a little experimentation I could have found another way without it.  So what I did was eliminate it.  In the examples I’m going to show in this thread that code won’t be there.  So you won’t be able to instantiate global C++ objects before main/WinMain is entered.  However, if you really want that code, Matt Pietrek provided it and it isn’t hard to replace it.  It'll add a about a kilobyte or so to your executable.

Next issue not really mentioned above from the Richter and Nasarre quote is command line arguments.  Any guesses how I’ve dealt with that?  You’ve guessed it.  They’re gone!  The elimination of the code to process command line arguments and pass them to main/wmain or WinMain/wWinMain involves about another kilobyte or so of startup code.  Again, I seldom use command line arguments.  Of course, they have their place (more so in the distant past, I think), and the code to deal with them can be found in Pietrek’s original code.   You can obtain it from there if you need it.  Here is Matt’s original mainCRTStartup() function which shows his call to his _ConvertCommandLineToArgcArgv(), which deals of course with the command line arguments, and his calls to atexitinit() and initterm() which are involved in setting up the global constructors…

Code: [Select]
// Modified version of the Visual C++ startup code.  Simplified to
// make it easier to read.  Only supports ANSI programs.
extern "C" void __cdecl mainCRTStartup(void)
{
    int mainret, argc;

    argc = _ConvertCommandLineToArgcArgv();
    // set up our minimal cheezy atexit table
    _atexit_init();
    // Call C++ constructors
    _initterm( __xc_a, __xc_z );
    mainret = main( argc, _ppszArgv, 0 );
    _DoExit();
    ExitProcess(mainret);
}

With the elimination of that code from the CRT startup what’s left?  Well, not much!  What’s left is what you’ll see below in my crt_con_a.cpp and  crt_con_w.cpp files, which are for, respectively, ansi or wide character initializations…

crt_con_a.cpp   // mainCRTStartup, or lack thereof, for asci builds

Code: [Select]
//========================================================================================
//                 Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                              By Fred Harris, January 2016
//
// cl crt_con_a.cpp /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
//========================================================================================
#include <windows.h>
#pragma comment(linker, "/defaultlib:kernel32.lib")
#pragma comment(linker, "/nodefaultlib:libc.lib")
#pragma comment(linker, "/nodefaultlib:libcmt.lib")
extern "C" int __cdecl  main();

extern "C" void __cdecl mainCRTStartup()
{
 int iReturn = main();
 ExitProcess(iReturn);
}

crt_con_w.cpp   // wmainCRTStartup, or lack thereof, for wide character builds

Code: [Select]
//========================================================================================
//                 Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                              By Fred Harris, January 2016
//
// cl crt_con_w.cpp /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
//========================================================================================
#include <windows.h>
#pragma comment(linker, "/defaultlib:kernel32.lib")
#pragma comment(linker, "/nodefaultlib:libc.lib")
#pragma comment(linker, "/nodefaultlib:libcmt.lib")
extern "C" int __cdecl  wmain();

extern "C" void __cdecl wmainCRTStartup()
{
 int iReturn = wmain();
 ExitProcess(iReturn);
}

If you don't mind getting your hands a little dirty at the command prompt working with this stuff, let's create a library for ourselves and put both those functins in it as well as this printf implementation below...

printf.cpp   // console output wide/narrow versions

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//                    cl printf.cpp /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>
#include <stdarg.h>
extern "C" int __cdecl  printf(const char*    format, ...);
extern "C" int __cdecl wprintf(const wchar_t* format, ...);
#pragma comment(linker, "/defaultlib:user32.lib") 


extern "C" int __cdecl printf(const char* format, ...)
{
 char szBuff[1024];
 DWORD cbWritten;
 va_list argptr;
 int retValue;
         
 va_start(argptr, format);
 retValue = wvsprintf(szBuff, format, argptr);
 va_end(argptr);
 WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), szBuff, retValue, &cbWritten, 0);

 return retValue;
}


extern "C" int __cdecl wprintf(const wchar_t* format, ...)
{
 wchar_t szBuffW[1024];
 char szBuffA[1024];
 int iChars,iBytes;
 DWORD cbWritten;
 va_list argptr;
           
 va_start(argptr, format);
 iChars = wvsprintfW(szBuffW, format, argptr);
 va_end(argptr);
 iBytes=WideCharToMultiByte(CP_ACP,0,szBuffW,iChars,szBuffA,1024,NULL,NULL);
 WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), szBuffA, iBytes, &cbWritten, 0);

 return iChars;
}

I'll provide a make file to make it easy, although you could simply run those three command line compilation strings included in the text headers for the above three functions to create *.obj files, then use lib.exe commands to create and add them to the library as I described in my last essay.  But in any case, here is the make file to run through nmake.  Save it to TCLib.mak...

Code: [Select]
PROJ       = TCLib

OBJS       = crt_con_a.obj crt_con_w.obj printf.obj
       
CC         = CL
CC_OPTIONS = /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN

$(PROJ).LIB: $(OBJS)
    LIB /NODEFAULTLIB /machine:x64 /OUT:$(PROJ).LIB $(OBJS)

.CPP.OBJ:
    $(CC) $(CC_OPTIONS) $<


Running the above make file using nmake produced this console output for me…

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test8>nmake TCLib.mak

Microsoft (R) Program Maintenance Utility Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        CL /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN crt_con_a.CPP
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

crt_con_a.CPP
        CL /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN crt_con_w.CPP
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

crt_con_w.CPP
        CL /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN printf.CPP
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

printf.CPP
        LIB /NODEFAULTLIB /machine:x64 /OUT:TCLib.LIB crt_con_a.obj crt_con_w.obj printf.obj
Microsoft (R) Library Manager Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

C:\Code\VStudio\VC15\LibCTiny\x64\Test8>

continued...


Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
Let's build some very simple executables to test this stuff.  But first I need to say a word about include files.  In C or C++ when you place include file names within '<IncludeFile.h>' brackets, that causes the preprocessor to look in specific places for include files, such as the INCLUDE environment path, or paths specified when the compiler was installed.  If, on the other hand, one places include file names within double quotes, i.e.,

Code: [Select]
#include "MyAppInclude.h"

...then the compiler looks for the include in the current directory first.  That is what we want.  I have had great difficulty using the Visual Studio C/C++ includes for this work involving eliminating the C runtime.  Especially with stdio.h, but to a lesser extent with string.h.  What happens is that something or other, either in those includes or includes included from those includes, triggers the loading of the C runtime.  Or there are definition conflicts with my replacement functions.  The decoration of C/C++ function declarations in the headers with bizarre macros is extreme.  In any case, to solve the problem I made my own replacements for stdio.h, string.h and tchar.h.  They are minimal to be sure.  stdio.h only has declarations for four functions.  Here are stdio.h and tchar.h. Put them in the directory where you are doing this work...   

stdio.h

Code: [Select]
// stdio.h
#ifndef stdio_h
#define stdio_h

extern "C" char __cdecl getchar();

extern "C" int  __cdecl printf(const char* format, ...);
extern "C" int  __cdecl wprintf(const wchar_t* format, ...);

extern "C" int  __cdecl sprintf(char* buffer, const char* format, ...);
extern "C" int  __cdecl swprintf(wchar_t* buffer, const wchar_t* format, ...);

#endif

tchar.h

Code: [Select]
// tchar.h
#ifndef tchar_h
#define tchar_h

#ifdef  _UNICODE
   typedef wchar_t  TCHAR;
   #define _tmain   wmain
   #define _T(x)    L## x
   #define _tprintf wprintf
#else
   typedef char     TCHAR;
   #define _tmain   main
   #define _T(x)    x
   #define _tprintf printf
#endif

#endif


For stdio.h, don’t include this like so…

Code: [Select]
#include <stdio.h>

…nor this way either…

Code: [Select]
#include <cstdio>

Rather, do this…

Code: [Select]
#include “stdio.h”

Before proceeding with our first test program, make sure you have TCLib.lib created, and within it are crt_con_a.obj, crt_con_w.obj, and printf.obj.  You can test that like so...

C:\........\>lib TCLib.lib /list /verbose  [ENTER]

....and it'll list those three objs.  Hopefully.  Also, you'll then need stdio.h and tchar.h in your working directory.  Here is our first test program to exercise what little we have so far…

Code: [Select]
// cl Ex01.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib
#define UNICODE         // 2,560 bytes x64 UNICODE
#define _UNICODE
#include <windows.h>
#include "tchar.h"

int _tmain()
{
 return 0;
}

That should give us the smallest x64 asci or Unicode program I know how to produce that still has a main() function, and its coming in 2,560 bytes for me with VC19 (from Visual Studio 2015).  That'll give us an idea of the baseline we're starting from.  Lets try adding a wprintf() to it to generate our classic “Hello, World!”…

Code: [Select]
// cl Ex02.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib
#define UNICODE          // 3,072 bytes x64 UNICODE
#define _UNICODE
#include <windows.h>
#include "stdio.h"
#include "tchar.h"

int _tmain()
{
 _tprintf(_T("Hello, World!\n"));
 return 0;
}

So we’re up to 3 kilobytes with wprintf or printf.  That cost us 512 bytes.  I don’t really consider console programs complete unless I have something to keep the console from closing in case the program was started from Explorer, so let’s add a getchar() to our TCLib.lib.  Here is that code.  You can compile it to an object file with the supplied command line string…

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//              cl getchar.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>

extern "C" char __cdecl getchar()
{
 DWORD nRead,dwConsoleMode;
 INPUT_RECORD ir[8];
 bool blnLoop=true;
 HANDLE hStdIn;
 char c=0;

 hStdIn=GetStdHandle(STD_INPUT_HANDLE);
 GetConsoleMode(hStdIn,&dwConsoleMode);
 SetConsoleMode(hStdIn,0);
 FlushConsoleInputBuffer(hStdIn);
 do
 {
    WaitForSingleObject(hStdIn,INFINITE);
    ReadConsoleInput(hStdIn,ir,8,&nRead);
    for(unsigned i=0;i<nRead;i++)
    {
        if(ir[i].EventType==KEY_EVENT && ir[i].Event.KeyEvent.bKeyDown==TRUE)
        {
           c=ir[i].Event.KeyEvent.uChar.AsciiChar;
           blnLoop=false;
        }
    }
 }while(blnLoop==true);
 SetConsoleMode(hStdIn,dwConsoleMode);

 return c;
}

That's pretty 'industrial strength' and not really a minimal implementation, but I wrote it so I'll live with it.  To add that to our TCLib.lib simply execute this line in your console window…

Code: [Select]
Lib TCLib.lib getchar.obj  [ENTER]

To make sure it worked just dump the contents of TCLib.lib like so…

Code: [Select]
Lib TCLib.lib /list /verbose

My console dump looks like so…

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test8>lib TCLib.lib /list /verbose
Microsoft (R) Library Manager Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

getchar.obj
crt_con_a.obj
crt_con_w.obj
printf.obj

C:\Code\VStudio\VC15\LibCTiny\x64\Test8>

And now with a getchar() to keep the console open (if started from Explorer), here is Ex03.cpp…

Code: [Select]
// cl Ex03.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib
#define UNICODE          // 3,072 bytes x64 UNICODE
#define _UNICODE
#include <windows.h>
#include "stdio.h"
#include "tchar.h"

int _tmain()
{
 _tprintf(_T("Hello, World!\n"));
 getchar();
 
 return 0;
}

So that doesn’t seem to have cost us anything, at least this session with VC19, as we’re still at 3 k for x64 UNICODE.

Now let’s move on to GUIs!  As mentioned in the Richter/Nasarre quote, we’ll need some kind of WinMain() for that.  You hopefully already have a TCLib.lib which works, so let’s just add a crt_win_a.cpp and a crt_win_w.cpp to it (after compiling to objs, of course).  Here is crt_win_a.cpp…

Code: [Select]
// crt_win_a.cpp
// cl crt_win_a.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#pragma comment(linker, "/defaultlib:kernel32.lib")
#pragma comment(linker, "/nodefaultlib:libc.lib")
#pragma comment(linker, "/nodefaultlib:libcmt.lib")
int WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR, int);

extern "C" void __cdecl WinMainCRTStartup(void)
{
 int iReturn = WinMain(GetModuleHandle(NULL),NULL,NULL,SW_SHOWDEFAULT);
 ExitProcess(iReturn);
}


And here is crt_win_w.cpp…


Code: [Select]
// crt_win_w.cpp
// cl crt_win_w.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#pragma comment(linker, "/defaultlib:kernel32.lib")
#pragma comment(linker, "/nodefaultlib:libc.lib")
#pragma comment(linker, "/nodefaultlib:libcmt.lib")
int WINAPI wWinMain(HINSTANCE, HINSTANCE, LPWSTR, int);

extern "C" void __cdecl wWinMainCRTStartup(void)
{
 int iReturn = wWinMain(GetModuleHandle(NULL),NULL,NULL,SW_SHOWDEFAULT);
 ExitProcess(iReturn);
}

Here is the command line session output of compiling both those files to object files, adding them to TCLib.lib, then dumping the contents of that library…

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test8>cl crt_win_a.cpp /D /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

crt_win_a.cpp

C:\Code\VStudio\VC15\LibCTiny\x64\Test8>cl crt_win_w.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

crt_win_w.cpp

C:\Code\VStudio\VC15\LibCTiny\x64\Test8>Lib TCLib.lib crt_win_a.obj crt_win_w.obj
Microsoft (R) Library Manager Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Replacing crt_win_a.obj
Replacing crt_win_w.obj

C:\Code\VStudio\VC15\LibCTiny\x64\Test8>lib TCLib.lib /list /verbose
Microsoft (R) Library Manager Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

crt_win_a.obj
crt_win_w.obj
getchar.obj
printf.obj
crt_con_w.obj
crt_con_a.obj


Now with WinMain and wWinMain entry points made known within the C Runtime Start up procedures we can try our luck with our GUI Form1.cpp….

Code: [Select]
// cl Form1.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib user32.lib gdi32.lib
// 3,072 Bytes x64 UNICODE
#define UNICODE   // /entry:wWinMainCRTStartup
#define _UNICODE
#include <windows.h>
#include "tchar.h"

LRESULT CALLBACK fnWndProc(HWND hwnd, unsigned int msg, WPARAM wParam, LPARAM lParam)
{
 if(msg==WM_DESTROY)
 {
    PostQuitMessage(0);
    return 0;
 }

 return (DefWindowProc(hwnd, msg, wParam, lParam));
}

int WINAPI _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevIns, LPTSTR lpszArgument, int iShow)
{
 WNDCLASSEX wc={};
 MSG messages;
 HWND hWnd;

 wc.lpszClassName = _T("Form1");
 wc.lpfnWndProc   = fnWndProc;
 wc.cbSize        = sizeof(WNDCLASSEX);
 wc.hInstance     = hInstance;
 wc.hbrBackground = (HBRUSH)COLOR_BTNSHADOW;
 RegisterClassEx(&wc);
 hWnd=CreateWindowEx(0,_T("Form1"),_T("Form1"),WS_OVERLAPPEDWINDOW|WS_VISIBLE,200,100,325,300,HWND_DESKTOP,0,hInstance,0);
 while(GetMessage(&messages,NULL,0,0))
 {
    TranslateMessage(&messages);
    DispatchMessage(&messages);
 }

 return messages.wParam;
}

And our luck’s not so good, ehh?  Here’s what I got….

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test8>cl Form1.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib user32.lib gdi32.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

Form1.cpp
Microsoft (R) Incremental Linker Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:Form1.exe
TCLib.lib
kernel32.lib
user32.lib
gdi32.lib
Form1.obj
Form1.obj : error LNK2019: unresolved external symbol memset referenced in function wWinMain
Form1.exe : fatal error LNK1120: 1 unresolved externals


If you are reading this Patrice, remember you mentioned trying this to zero out a WNDCLASSEX struct…

Code: [Select]
WNDCLASSEX wc={};

Well, now you know how {} on a struct is implemented!  :)  It’s a bit disconcerting to realize the compiler isn’t doing exactly what you would expect, isn’t it?

But not to worry!  We can get around that.  In the above command line compilation string for Form1…


cl Form1.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib user32.lib gdi32.lib

…we are telling cl to optimize for size and not put any debugging symbols in the executable.  Lets remove the /O1 and /Os and in its place substitute /Oi, which tells the compiler to use compiler intrinsics when possible…

https://en.wikipedia.org/wiki/Intrinsic_function

And this from Microsoft…

https://msdn.microsoft.com/en-us/library/26td21ds.aspx

Our cl string then becomes this…


cl Form1.cpp /Oi /GS- /link TCLib.lib kernel32.lib user32.lib gdi32.lib

…and we succeed!  Which is an understatement.  Look at the size of this x64 UNICODE executable.  Its 3,072 bytes!  Run it and you’ll see it works just fine.

Next thing to do is get this all working with classes, particularly my String Class.

To be continued………

Offline James C. Fuller

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 595
  • User-Rate: +11/-8
Quote
If you are reading this Patrice, remember you mentioned trying this to zero out a WNDCLASSEX struct…
WNDCLASSEX wc={};

Fred,
  I believe that should be:
Code: [Select]
WNDCLASSEX wc={0};

James

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
Thanks Jim, you're likely right!  Alas, I need to beef up my knowledge of the usage of that construct.  I picked that up from browsing C++ forums I think.  Likely I fell into the usage of not putting the zero in there by accident.  Never saw that described in any of my various C or C++ books. 

Putting it in doesn't change anything though in the program.  It'll still error out on a missing memset call.  That memset thing is kind of interesting.  It seems to be some kind of primal force or something.  When I first ran into problems with it on my post a couple weeks ago, I tried substituting a for loop for it, but as I said, the optimizer substituted a memset call for the for loop which still resulted in the linker error missing memset.  Then I decided to get tricky and call ZeroMemory instead, which, if my memory serves me right (and it might not), is a Windows function - not C Runtime.  But got the same missing memset linker error on that too!  So that's why I'm saying it seems to be some kind of primal thing in the operating system.  All kinds of stuff results in memset calls. 

But the odd thing about it that kind of doesn't make sense is that it is a compiler intrinsic, and the compiler should know about it, but doesn't seem to in some instances.  In researching the topic I discovered to my pleasure that some of the most important string functions were intrinsics.  Here is a list of them...

memset
wmemset
strcpy
wcscpy
strcat
wcscat
strcmp
wcscmp

So I didn't need to implement them in my versions of LibCTiny.lib because the compiler always had them.  You do need function prototypes though.  But the memset is different.  It seems to be needed in some instances. 

Offline James C. Fuller

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 595
  • User-Rate: +11/-8
Fred,
Not sure this is relevant to any of your work but to help stop optimizations check out Volatile variables.

James

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
As I mentioned before, one really needs the features of C++ to do application programming – in my opinion.   C just doesn’t have a good enough feature set.   So I need to get this stuff working in C++.  And it actually does already.  With what I’ve provided so far one can create classes with no problem and no bloat.  But I had a great deal of difficulty with getting my String Class working with VC19 from Visual Studio 2015 in this most recent work of mine in updating it to include UNICODE capabilities.  I hit a problem similar to my _fltused debacle from my post several weeks ago.  Lost three or four days on it actually, same as with that miserable _fltused thing.  The problem involved my String::Parse() member function, which is somewhat unusual in my String Class, in that it is the only member which takes as a parameter a pointer to an array of String objects.  Here is the declaration of it…

Code: [Select]
// Parses this based on delimiter.  Must pass in 1st parameter String* to sufficient number of Strings
void Parse(String* pStr, TCHAR delimiter, size_t iParseCount);

So rather than just presenting my finished String Class I’d like to bore my readers with the details of my struggle with this, in the hope it may help someone with similar difficulties.  After that I’ll provide my whole String Class in TCHAR form so it can be used in x86/x64 in UNICODE or ASCI.  Along the way I’ll provide both the String Class and a C based implementation of the String::Parse() algorithm for comparison purposes. 

To begin with, my implementation of String::Parse() is based on the PowerBASIC model.  Assume we have this WString…

Code: [Select]
Local strLine As Wstring
StrLine = “Zero, One, Two, Three, Four, Five”

Then a PowerBASIC Console Compiler program to parse strLine looks like this with the expected output afterwards…

Code: [Select]
#Compile Exe
#Dim All

Function PBMain() As Long
  Local iParseCount As Long
  Local strLine As WString
  Local pStrs() As WString
  Register i As Long

  strLine="Zero, One, Two, Three, Four, Five"
  iParseCount=ParseCount(strLine,",")
  Redim pStrs(iParseCount) As WString
  Parse strLine, pStrs(), ","
  For i=0 To UBound(pStrs, 1) -1
    Console.Print LTrim$(pStrs(i))
  Next i
  Erase pStrs()
  Waitkey$

  PBMain=0
End Function

#if 0

Zero
One
Two
Three
Four
Five

#EndIf
               
The way it works is to first determine the number of strings delimited by some delimiter.  PowerBASIC has the ParseCount() function for that.  In the case above it of course returns 6.  Then for the Parse statement to work it needs a dynamically allocated string array with enough elements to contain the number of strings as determined by ParseCount.  The Parse statement takes as the 1st parameter the string to parse and the 2nd parameter the dynamically allocated string array.  The last parameter is the delimiter; optional if it is a comma.  But I stuck it in above anyway for clarity. 

So in converting this to my C++ String Class it seemed necessary to use a String* fabrication in lieu of the PowerBASIC…

Code: [Select]
Redim pStrs(iParseCount) As WString

…syntax with Redim.  And to dynamically allocate an array of String objects the C++ new operator would be used, something like this…

Code: [Select]
String* pStrs = new String[iParseCount];

Accepting all that, a String::Parse(….) member function could have a declaration something along these lines…

Code: [Select]
void Parse(String* pStr, TCHAR delimiter);

or this…

Code: [Select]
void Parse(String* pStr, TCHAR delimiter, size_t iParseCount);


Notice I have it declared as returning void, and the String it operates upon is ‘this’, i.e., the one with the delimited fields, which it doesn’t alter in any way. 

Actually, for many years I used the top implementation with only two parameters.  Just recently I came up with a slightly better algorithm that saved a couple lines of code by passing into the member the iParseCount number of strings, which one has to know anyway before calling String::Parse() or even PowerBASIC Parse, so the memory allocation can be made.

So lets see what happens when we try to implement this with Visual Studio 2015’s VC compiler version 19.  First we’ll create a stripped down String Class with only enough functionality to support String::Parse().  Then we’ll add String::ParseCount() and String::Parse() to it.  The only three absolutely necessary String Class members which must be implemented for String::Parse() to work are an un-initialized constructor, an operator= member, and a destructor.  An un-initialized constructor is necessary because when our String* is set equal to the return from a call to…

new String[iParseCount]

…the un-initialized constructor will be called iParseCount times to create each empty String object.  The operator= member is needed because it defines the meaning of the equal sign when an assignment is made for each parsed string where a String object is on the left of the equal sign and a character string is on the right.  The destructor is necessary of course to release memory allocated to hold each string.

     So here’s the code.  You’ll need to make another directory for all this or delete what I’ve previously given you, as I’m providing all new files including a make file to add a lot more *.obj files to our Lib.  Here first is the make file to be saved as TCLib.mak and run with nmake…

Code: [Select]
PROJ       = TCLib

OBJS       = crt_con_a.obj crt_con_w.obj crt_win_a.obj crt_win_w.obj memset.obj newdel.obj printf.obj \
             sprintf.obj _strnicmp.obj strncpy.obj strncmp.obj _strrev.obj strcat.obj strcmp.obj \
             strcpy.obj strlen.obj getchar.obj alloc.obj alloc2.obj allocsup.obj
       
CC         = CL
CC_OPTIONS = /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN

$(PROJ).LIB: $(OBJS)
    LIB /NODEFAULTLIB /machine:x64 /OUT:$(PROJ).LIB $(OBJS)

.CPP.OBJ:
    $(CC) $(CC_OPTIONS) $<

By the way, I’ll attach all this in a zip file for those who are members here.  I don’t have to reproduce these files…

crt_con_a.cpp
crt_con_w.cpp
crt_win_a.cpp
crt_win_w.cpp
printf.cpp
getchar.cpp

… as the ones I gave in my last posting of this thread are what I’m now using.  You can look back to find them.  So I’ll start with the String.h file, which is my abbreviated version of the real string.h….

string.h

Code: [Select]
// String.h
#ifndef string_h
#define string_h

extern "C" size_t   __cdecl strlen(const char* pStr);
extern "C" size_t   __cdecl wcslen(const wchar_t* pStr);

extern "C" char*    __cdecl strcpy(char* strDestination, const char* strSource);
extern "C" wchar_t* __cdecl wcscpy(wchar_t* strDestination, const wchar_t* strSource);

extern "C" char*    __cdecl strcat(char* strDest, const char* strSource);
extern "C" wchar_t* __cdecl wcscat(wchar_t* strDest, const wchar_t* strSource);

extern "C" int      __cdecl strcmp(const char* string1, const char* string2);
extern "C" int      __cdecl wcscmp(const wchar_t* string1, const wchar_t* string2);

extern "C" int      __cdecl _strnicmp(const char* str1, const char* str2, size_t count);
extern "C" int      __cdecl _wcsnicmp(const wchar_t* str1, const wchar_t* str2, size_t count);

extern "C" char*    __cdecl strncpy(char* strDest, const char* strSource, size_t iCount);   
extern "C" wchar_t* __cdecl wcsncpy(wchar_t* strDest, const wchar_t* strSource, size_t iCount); 

extern "C" char*    __cdecl _strrev(char* pStr);
extern "C" wchar_t* __cdecl _wcsrev(wchar_t* pStr);
#endif

Now for stdlib.h

Code: [Select]
// stdlib.h
#ifndef stdlib_h
#define stdlib_h
   #define NULL 0
   extern "C" void* __cdecl malloc(size_t size);
   extern "C" void  __cdecl free(void* pMem);
#endif

Now tchar.h…

Code: [Select]
// tchar.h
#ifndef tchar_h
   #define tchar_h
   #ifdef  _UNICODE
      typedef wchar_t     TCHAR;
      #define _T(x)       L## x
      #define _tmain      wmain
      #define _tWinMain   wWinMain
      #define _tprintf    wprintf
      #define _stprintf   swprintf
      #define _tcslen     wcslen
      #define _tcscpy     wcscpy
      #define _tcscat     wcscat
      #define _tcsncpy    wcsncpy
      #define _tcsncmp    wcsncmp
      #define _tcsnicmp   _wcsnicmp
      #define _tcsrev     _wcsrev
      #define _ttoi64     _wtoi64
   #else
      typedef char        TCHAR;
      #define _T(x)       x
      #define _tmain      main
      #define _tWinMain   WinMain
      #define _tprintf    printf
      #define _stprintf   sprintf
      #define _tcslen     strlen
      #define _tcscpy     strcpy
      #define _tcscat     strcat
      #define _tcsncpy    strncpy
      #define _tcsncmp    strncmp
      #define _tcsnicmp   _strnicmp
      #define _tcsrev     _strrev
      #define _ttoi64     _atoi64
   #endif
#endif

You should have stdio.h from my last postings.  And don’t forget to get it or you’ll have problems.  Here is _strnicmp.cpp, which contains _strnicmp.cpp and _wcsnicmp.cpp…

_strnicmp and _wcsnicmp

Code: [Select]
// _strnicmp.cpp
//==============================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//     cl _strnicmp.cpp /D "_CRT_SECURE_NO_WARNINGS" /c /W3 /DWIN32_LEAN_AND_MEAN
//==============================================================================================
#include <windows.h>
#include <malloc.h>
#include "string.h"

extern "C" int __cdecl _strnicmp(const char* str1, const char* str2, size_t count)
{
 size_t iLen1=strlen(str1);
 size_t iLen2=strlen(str2);
 if(count>iLen1)
    return -1;
 if(count>iLen2)
    return 1;
 char* pStr1=(char*)malloc(count+1);
 char* pStr2=(char*)malloc(count+1);
 strncpy(pStr1,str1,count);
 strncpy(pStr2,str2,count);
 pStr1[count]=0;
 pStr2[count]=0;
 int iReturn=lstrcmpiA(pStr1,pStr2);
 free(pStr1);
 free(pStr2);

 return iReturn;
}

extern "C" int __cdecl _wcsnicmp(const wchar_t* str1, const wchar_t* str2, size_t count)
{
 size_t iLen1=wcslen(str1);
 size_t iLen2=wcslen(str2);
 if(count>iLen1)
    return -1;
 if(count>iLen2)
    return 1;
 wchar_t* pStr1=(wchar_t*)malloc(count*2+2);
 wchar_t* pStr2=(wchar_t*)malloc(count*2+2);
 wcsncpy(pStr1,str1,count);
 wcsncpy(pStr2,str2,count);
 pStr1[count]=0;
 pStr2[count]=0;
 int iReturn=lstrcmpiW(pStr1,pStr2);
 free(pStr1);
 free(pStr2);

 return iReturn;
}

_strrev and _wcsrev.  Filename is _strrev.cpp

Code: [Select]
//========================================================
// Developed As An Addition To Matt Pietrek's LibCTiny.lib
//              By Fred Harris, January 2016
//
//     cl _strrev.cpp /Oi /c /W3 /DWIN32_LEAN_AND_MEAN
//========================================================
#include <windows.h>
#include "string.h"

extern "C" char* __cdecl _strrev(char* pStr)
{
 size_t iLen,iHalf;
 char a,b;

 iLen=strlen(pStr), iHalf=iLen/2;
 for(size_t i=0; i<iHalf; i++)
 {
     a=pStr[i], b=pStr[iLen-i-1];
     pStr[i]=b, pStr[iLen-i-1]=a;
 }

 return pStr;
}

extern "C" wchar_t* __cdecl _wcsrev(wchar_t* pStr)
{
 size_t iLen,iHalf;
 wchar_t a,b;

 iLen=wcslen(pStr), iHalf=iLen/2;
 for(size_t i=0; i<iHalf; i++)
 {
     a=pStr[i], b=pStr[iLen-i-1];
     pStr[i]=b, pStr[iLen-i-1]=a;
 }

 return pStr;
}

alloc.cpp (from Matt Pietrek, Microsoft Systems Journal)

Code: [Select]
// cl alloc.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#include <malloc.h>

extern "C" void* __cdecl malloc(size_t size)
{
 return HeapAlloc(GetProcessHeap(), 0, size);
}

extern "C" void __cdecl free(void* pMem)
{
 HeapFree(GetProcessHeap(), 0, pMem);
}

alloc2.cpp (from Matt Pietrek, Microsoft Systems Journal)

Code: [Select]
// cl alloc2.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#include <malloc.h>

extern "C" void* __cdecl realloc(void* pMem, size_t size)
{
 if(pMem)
    return HeapReAlloc(GetProcessHeap(), 0, pMem, size);
 else
    return HeapAlloc(GetProcessHeap(), 0, size);
}

extern "C" void* __cdecl calloc(size_t nitems, size_t size)
{
 return HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, nitems * size);
}


allocsup.cpp (from Matt Pietrek, Microsoft Systems Journal)
Code: [Select]
// cl allocsup.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#include <malloc.h>

extern "C" void* __cdecl _nh_malloc(size_t size, int nhFlag)
{
 return HeapAlloc(GetProcessHeap(), 0, size);
}

extern "C" size_t __cdecl _msize(void* pMem)
{
 return HeapSize(GetProcessHeap(), 0, pMem);
}

memset.cpp

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//                     cl memset.cpp /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================

extern "C" void* __cdecl memset(void* p, int c, size_t count)
{
 char* pCh=(char*)p;
 for(size_t i=0; i<count; i++)
     pCh[i]=(char)c;
 return p;
}

newdel.cpp

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//
//                          LIBCTINY -- Matt Pietrek 2001
//                           MSDN Magazine, January 2001
//
//                              With Help From Mike_V
//                       
//                           By Fred Harris, January 2016
//
//                    cl newdel.cpp /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>

void* __cdecl operator new(size_t s)
{
 return HeapAlloc(GetProcessHeap(), 0, s);
}

void  __cdecl operator delete(void* p)
{
 HeapFree(GetProcessHeap(), 0, p);
}

void* operator new [] (size_t s)
{
 return HeapAlloc(GetProcessHeap(), 0, s);
}

void operator delete [] (void* p)
{
 HeapFree(GetProcessHeap(), 0, p);
}

sprintf.cpp

Code: [Select]
//=============================================================
//   Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                By Fred Harris, February 2016
//
// cl sprintf.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN   
//=============================================================
// cl sprintf.cpp /O1 /Os /GS- /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#include <stdarg.h>
#include "stdio.h"
#define EOF (-1)
#pragma comment(linker, "/defaultlib:user32.lib")

extern "C" int __cdecl sprintf(char* buffer, const char* format, ...)
{
 va_list argptr;
 int retValue;
           
 va_start(argptr, format);
 retValue = wvsprintfA(buffer, format, argptr);
 va_end(argptr);

 return retValue;
}

extern "C" int __cdecl swprintf(wchar_t* buffer, const wchar_t* format, ...)
{
 va_list argptr;
 int retValue;
         
 va_start(argptr, format);
 retValue = wvsprintfW(buffer, format, argptr);
 va_end(argptr);

 return retValue;
}

strcat.cpp

Code: [Select]
//===================================================================
//   Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                By Fred Harris, January 2016
//
//         cl strcat.cpp /c /W3 /DWIN32_LEAN_AND_MEAN
//===================================================================
#include <windows.h>

extern "C" char* __cdecl strcat(char* strDest, const char* strSource)   
{                                           // 3rd param is size_t
 return lstrcatA(strDest, strSource);
}

extern "C" wchar_t* __cdecl wcscat(wchar_t* strDest, const wchar_t* strSource)   
{                                           // 3rd param is size_t
 return lstrcatW(strDest, strSource);
}

strcmp.cpp

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//                      cl strcmp.cpp /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>

extern "C" int __cdecl strcmp(const char* string1, const char* string2)   
{
 return lstrcmpA(string1,string2);
}

extern "C" int __cdecl wcscmp(const wchar_t* string1, const wchar_t* string2)   
{
 return lstrcmpW(string1,string2);
}

strcpy.cpp

Code: [Select]
// cl strcpy.cpp /c /W3 /DWIN32_LEAN_AND_MEAN
#include <windows.h>
#include <string.h>

extern "C" char* __cdecl strcpy(char* strDestination, const char* strSource)
{
 return lstrcpyA(strDestination, strSource);
}

extern "C" wchar_t* __cdecl wcscpy(wchar_t* strDestination, const wchar_t* strSource)
{
 return lstrcpyW(strDestination, strSource);
}

strlen.cpp

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//                      cl strlen.cpp /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>

extern "C" size_t __cdecl strlen(const char* strSource)
{
 return lstrlenA(strSource);
}

extern "C" size_t __cdecl wcslen(const wchar_t* strSource)
{
 return lstrlenW(strSource);
}

strncmp.cpp

Code: [Select]
//===============================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//       cl strncmp.cpp /D "_CRT_SECURE_NO_WARNINGS" /c /W3 /DWIN32_LEAN_AND_MEAN
//===============================================================================================
#include <windows.h>
#include "string.h"
#include <malloc.h>

extern "C" int __cdecl strncmp(const char* str1, const char* str2, size_t count)
{
 size_t iLen1=strlen(str1);
 size_t iLen2=strlen(str2);
 if(count>iLen1)
    return -1;
 if(count>iLen2)
    return 1;
 char* pStr1=(char*)malloc(count+1);
 char* pStr2=(char*)malloc(count+1);
 strncpy(pStr1,str1,count);
 strncpy(pStr2,str2,count);
 pStr1[count]=0;
 pStr2[count]=0;
 int iReturn=strcmp(pStr1,pStr2);
 free(pStr1);
 free(pStr2);

 return iReturn;
}

extern "C" int __cdecl wcsncmp(const wchar_t* str1, const wchar_t* str2, size_t count)
{
 size_t iLen1=wcslen(str1);
 size_t iLen2=wcslen(str2);
 if(count>iLen1)
    return -1;
 if(count>iLen2)
    return 1;
 wchar_t* pStr1=(wchar_t*)malloc(count*2+2);
 wchar_t* pStr2=(wchar_t*)malloc(count*2+2);
 wcsncpy(pStr1,str1,count);
 wcsncpy(pStr2,str2,count);
 pStr1[count]=0;
 pStr2[count]=0;
 int iReturn=wcscmp(pStr1,pStr2);
 free(pStr1);
 free(pStr2);

 return iReturn;
}

strncpy.cpp

Code: [Select]
//=====================================================================================
//               Developed As An Addition To Matt Pietrek's LibCTiny.lib
//                            By Fred Harris, January 2016
//
//                 cl strncpy.cpp /O1 /Os /c /W3 /DWIN32_LEAN_AND_MEAN
//=====================================================================================
#include <windows.h>
#include "string.h"


extern "C" char* __cdecl strncpy(char* strDest, const char* strSource, size_t iCount)   
{                                              // 3rd param is size_t
 size_t iLenSrc=strlen(strSource);             // strlen returns size_t                                           
 lstrcpynA(strDest,strSource,(int)iCount);     // lstrcpyn wants an int here for 3rd param                                           
 strDest[iCount-1]=strSource[iCount-1];        // so try cast
 if(iCount>iLenSrc)
 {
    for(size_t i=iLenSrc; i<iCount; i++)
        strDest[i]=0;
 }

 return strDest;
}


extern "C" wchar_t* __cdecl wcsncpy(wchar_t* strDest, const wchar_t* strSource, size_t iCount)   
{                                           // 3rd param is size_t
 size_t iLen=wcslen(strSource);             // strlen returns size_t                                           
 lstrcpynW(strDest,strSource,(int)iCount);  // lstrcpyn wants an int here for 3rd param                                           
 strDest[iCount-1]=strSource[iCount-1];     // so try cast
 if(iCount>iLen)
 {
    for(size_t i=iLen; i<iCount; i++)
        strDest[i]=0;
 }

 return strDest;


And that should be it!  You’ll notice for many of the files there are two functions contained within; one for ASCI and the other for UNICODE.   If you’re trying to duplicate my work here I realize it will be difficult to keep track of all these files.  To make sure I had them all I listed them like so…

Code: [Select]
crt_con_a.cpp  ck
crt_con_w.cpp  ck
crt_win_a.cpp  ck
crt_win_w.cpp  ck
printf.cpp     ck
getchar.cpp    ck
stdio.h
string.h
stdlib.h
tchar.h
_strnicmp      ck
_strrev        ck
alloc.cpp      ck
alloc2.cpp     ck
allocsup.cpp   ck
memset.cpp     ck
newdel.cpp     ck
sprintf.cpp    ck
strcat.cpp     ck
strcmp.cpp     ck
strcpy.cpp     ck
strlen.cpp     ck
strncmp.cpp    ck
strncpy.cpp    ck


... and checked that list against the listing in TCLib.mak.  If you miss any nmake will error out when you run it on TCLib.mak, and will report the missing file name to you.

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
Now we’ll start with the minimal version of my String Class with everything in it to support just String::Parse().  First Strings.h…

Code: [Select]
// Strings.h
#ifndef Strings_h
#define Strings_h

#ifndef ssize_t
typedef SSIZE_T ssize_t;             // ssize_t is defined in GCC, but isn't defined in VC9-15, but rather SSIZE_T.  For symetry we'll define it.
#endif

#define MINIMUM_ALLOCATION     16    // allocate at least this for every String
#define EXPANSION_FACTOR        2    // repeated concatenations will keep doubling buffer
#define NUMERIC_CONVERSIONS          // allows direct assignment of numeric values to Strings, e.g., String s(3.14159) or s=3.14159;
#define FORMAT_MONEY                 // put commas every three places, rounding, left/right justification, specify field sizes (padding), etc.
#define CONSOLE_OUTPUT               // support console output, i.e., enable String::Print()
#define                       x64
//#define                   debug

class String
{
 public:                                           // Constructors (8)
 String();                                         // Uninitialized Constructor~String();
 String& operator=(const TCHAR* pStr);             // Assign a character string to a String Object, e.g.,  String s1 = "Compile Without Compromise";
 #ifdef CONSOLE_OUTPUT
    void Print(bool blnCrLf);                      // Outputs String with or without CrLf
    void Print(const TCHAR* pText, bool blnCrLf);  // Parameter #1 - leading text literal; Parameter #2 - with/without CrLf
 #endif
 int ParseCount(const TCHAR delimiter);                         // Returns count of delimited fields as specified by char delimiter, i.e., comma, space, tab, etc.
 void Parse(String* pStr, TCHAR delimiter, size_t iParseCount); // Parses this based on delimiter.  Must pass in 1st parameter String* to sufficient number of Strings
 void LTrim();                                                  // Removes leading white space by modifying existing String
 TCHAR* lpStr();                                                // Same as std::string.c_str().  Returns pointer to underlying Z String
 ~String();                                                     //String Destructor

 private:
 TCHAR*    lpBuffer;
 size_t    iLen;
 size_t    iCapacity;
};
 
#endif
// End Strings.h

Now Strings.cpp

Code: [Select]
// Strings.cpp
#define UNICODE
#define _UNICODE
#include  <windows.h>
#include  "string.h"
#include  "stdio.h"
#include  "Strings.h"
#include  "tchar.h"


String::String()
{
 #ifdef debug
 _tprintf(_T("Entering String Uninitialized Constructor!\n"));
 #endif
 this->lpBuffer=new TCHAR[16];
 this->lpBuffer[0]=0;
 this->iLen=0;
 this->iCapacity=15;
 #ifdef debug
 _tprintf(_T("  this = %p\n"),this);
 _tprintf(_T("Leaving String Uninitialized Constructor!\n\n"));
 #endif
}


String& String::operator=(const TCHAR* pStr)  // Assign TCHAR* to String
{
 #ifdef debug
 _tprintf(_T("Entering String::operator=(TCHAR*)\n"));
 _tprintf(_T("  this            = %p\n"),this);
 _tprintf(_T("  pStr            = %s\n"),pStr);
 _tprintf(_T("  _tcslen(pStr)   = %Iu\n"),_tcslen(pStr));
 _tprintf(_T("  this->iCapacity = %Iu\n"),this->iCapacity);
 #endif
 size_t iNewLen=_tcslen(pStr);
 if(iNewLen>this->iCapacity)
 {
    #ifdef debug
    _tprintf(_T("  Had To Reallocate!\n"));
    #endif
    delete [] this->lpBuffer;
    int iNewSize=(iNewLen*EXPANSION_FACTOR/16+1)*16;
    this->lpBuffer=new TCHAR[iNewSize];
    this->iCapacity=iNewSize-1;
    #ifdef debug
    _tprintf(_T("  this->iCapacity = %Iu\n"),this->iCapacity);
    #endif
 }
 _tcscpy(this->lpBuffer,pStr);
 this->iLen=iNewLen;
 #ifdef debug
 _tprintf(_T("  this->lpBuffer  = %s\n"),this->lpBuffer);
 _tprintf(_T("Leaving String::operator=(TCHAR*)\n\n"));
 #endif
   
 return *this;
}


int String::ParseCount(const TCHAR delimiter)   //returns one more than # of
{                                               //delimiters so it accurately
 int iCtr=0;                                    //reflects # of strings delimited
 TCHAR* p;                                      //by delimiter.

 p=this->lpBuffer;
 while(*p)
 {
   if(*p==delimiter)
      iCtr++;
   p++;
 }

 return ++iCtr;
}


void String::Parse(String* pStr, TCHAR delimiter, size_t iParseCount)
{
 TCHAR* pBuffer=new TCHAR[this->iLen+1];
 if(pBuffer)
 {
    TCHAR* p=pBuffer;
    TCHAR* c=this->lpBuffer;
    while(*c)
    {
       if(*c==delimiter)
          *p=0;
       else
          *p=*c;
       p++, c++;
    }
    *p=0, p=pBuffer;
    for(size_t i=0; i<iParseCount; i++)
    {
        pStr[i]=p;
        p=p+pStr[i].iLen+1;
    }
    delete [] pBuffer;
 }
}


void String::LTrim()
{
 size_t iCt=0;

 for(size_t i=0; i<this->iLen; i++)
 {
     if(this->lpBuffer[i]==9||this->lpBuffer[i]==10||this->lpBuffer[i]==13||this->lpBuffer[i]==32)
        iCt++;
     else
        break;
 }
 if(iCt)
 {
    for(size_t i=iCt; i<=this->iLen; i++)
        this->lpBuffer[i-iCt]=this->lpBuffer[i];
 }
 this->iLen=this->iLen-iCt;
}


TCHAR* String::lpStr()
{
 return this->lpBuffer;
}


#ifdef CONSOLE_OUTPUT
   void String::Print(bool blnCrLf)
   {
    _tprintf(_T("%s"),this->lpBuffer);
    if(blnCrLf)
       _tprintf(_T("\n"));
   }

   
   void String::Print(const TCHAR* pStr, bool blnCrLf)
   {
    _tprintf(_T("%s%s"),pStr,lpBuffer);
    if(blnCrLf)
       _tprintf(_T("\n"));
   }
#endif


String::~String()
{
 #ifdef debug
 _tprintf(_T("Entering String Destructor!\n"));
 _tprintf(_T("  this = %p\n"),this);
 #endif
 delete [] this->lpBuffer;
 #ifdef debug
 _tprintf(_T("Leaving String Destructor!\n\n"));
 #endif
}
// End Strings.cpp

And here is Test1.cpp which is essentially a C++ version of my PowerBASIC Console Compiler program presented earlier…

Code: [Select]
// cl Test1.cpp Strings.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib      << Won't Work With TCLib.lib
#define UNICODE
#define _UNICODE      // 122,880 bytes
#include <windows.h>  //   4,608 bytes
#include "stdio.h"    // =============   
#include "tchar.h"    // 118,272 bytes     26.66 times smaller
#include "Strings.h"

int _tmain()
{
 int iParseCount=0;
 String* pStrs=NULL;
 String s1;
 
 s1=_T("Zero, One, Two, Three, Four, Five");
 s1.Print(_T("s1 = "),true);
 iParseCount=s1.ParseCount(_T(','));
 _tprintf(_T("iParseCount = %d\n\n"),iParseCount);
 pStrs=new String[iParseCount];
 s1.Parse(pStrs, _T(','), iParseCount);
 for(int i=0; i<iParseCount; i++)
 {
     pStrs[i].LTrim();
     pStrs[i].Print(true);
 }
 delete [] pStrs;
 getchar();
 
 return 0;
}

If you attempt to run that through cl after you’ve satisfactorily built TCLib.lib, here’s the compiler output you’ll get…

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test9>cl Test1.cpp Strings.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

Test1.cpp
Strings.cpp
Generating Code...
Microsoft (R) Incremental Linker Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:Test1.exe
TCLib.lib
kernel32.lib
Test1.obj
Strings.obj
Test1.obj : error LNK2019: unresolved external symbol "void __cdecl operator delete(void *,unsigned __int64)" (??3@YAXPEAX_K@Z) referenced in function "public: void * __cdecl String::`vector deleting destructor'(unsigned int)" (??_EString@@QEAAPEAXI@Z)
Test1.obj : error LNK2019: unresolved external symbol "void __cdecl operator delete[](void *,unsigned __int64)" (??_V@YAXPEAX_K@Z) referenced in function "public: void * __cdecl String::`vector deleting destructor'(unsigned int)" (??_EString@@QEAAPEAXI@Z)

Test1.exe : fatal error LNK1120: 2 unresolved externals


Two pretty evil looking linker errors involving unresolved externals is what I got.  The function signatures it was tripping over are these…

Code: [Select]
void __cdecl operator delete    (void*, unsigned __int64)
void __cdecl operator delete [] (void*, unsigned __int64)

And then is this even worse gibberish…

Code: [Select]
void* __cdecl String::`vector deleting destructor'(unsigned int)" (??_EString@@QEAAPEAXI@Z)
void* __cdecl String::`vector deleting destructor'(unsigned int)  (??_EString@@QEAAPEAXI@Z)

And realize that I’ve been using essentially this same code for over ten years starting out first with VC6 circa 1998 or so, then Visual Studio 2003, and 2008.  And for GCC starting with their initial port of GCC for Windows around 2000 or so up through their most recent version.  And I’d never seen any errors like these.  To be perfectly honest, I’m not even sure at this point how this issue allowed me to succeed several weeks ago when I posted my first asci version of all this.  You’ll note the linker is complaining about delete and delete [] functions with two parameters; a void* and an unsigned _int64.  If for some unknown reason it needs delete and delete [] functions as above with two parameters I can see why it is complaining because none of my versions of delete have two; just one parameter like so…

Code: [Select]
void* __cdecl operator new(size_t s)
{
 return HeapAlloc(GetProcessHeap(), 0, s);
}

void  __cdecl operator delete(void* p)
{
 HeapFree(GetProcessHeap(), 0, p);
}

void* operator new [] (size_t s)
{
 return HeapAlloc(GetProcessHeap(), 0, s);
}

void operator delete [] (void* p)
{
 HeapFree(GetProcessHeap(), 0, p);
}

And the fact that this program was working perfectly with VC15 from Visual Studio 2008 led me to suspect that the ongoing recent changes to the C++ language standards were somehow the cause of all this.  And I know for sure that my biggest failing as a programmer is not keeping up with the ever changing versions of things everybody is always cranking out.  I hate so to upgrade anything!  So with those thoughts in mind I sank a lot of time researching the most up to date usages of new, operator new, delete, and operator delete for anything that might be a clue or anything that I didn’t recognize which might be the cause of this.  But I didn’t find anything.  Whatever changes were made recently to the C++ standards didn’t seem to affect these memory release and destruction mechanisms.  At least I didn't see anything which rendered the old usages useless.   I even tried creating those functions in my own code to see if that would help.  What happened there is that it stopped the linker errors but crashed the program upon being run.

Thinking back on how much time I had lost on that _fltused debacle of several weeks past, where I found an immediate answer as soon as I searched the internet on it, I tried that too.  But unfortunately nothing was jumping out at me as being a solution.  That was kind of the last straw.  How was I ever going to fix this?

My next thought was that the problem must somehow be related to changes made to the compiler to support new features, and that Microsoft had a lot of compiler and linkage switches to help support various vintages of code.  Perhaps there was something there to turn off recent changes?  Here is the ‘Languages’ sub-heading from a dump of the switches obtained from a cl /? Invocation…

Code: [Select]
                                -LANGUAGE-

/Zi enable debugging information        /Z7 enable old-style debug info
/Zp[n] pack structs on n-byte boundary  /Za disable extensions
/Ze enable extensions (default)         /Zl omit default library name in .OBJ
/Zs syntax check only                   /vd{0|1|2} disable/enable vtordisp
/vm<x> type of pointers to members
/Zc:arg1[,arg2] C++ language conformance, where arguments can be:
  forScope[-]           enforce Standard C++ for scoping rules
  wchar_t[-]            wchar_t is the native type, not a typedef
  auto[-]               enforce the new Standard C++ meaning for auto
  trigraphs[-]          enable trigraphs (off by default)
  rvalueCast[-]         enforce Standard C++ explicit type conversion rules
  strictStrings[-]      disable string-literal to [char|wchar_t]*
                        conversion (off by default)
  implicitNoexcept[-]   enable implicit noexcept on required functions
  threadSafeInit[-]     enable thread-safe local static initialization
  inline[-]             remove unreferenced function or data if it is
                        COMDAT or has internal linkage only (off by default)
  sizedDealloc[-]       enable C++14 global sized deallocation
                        functions (on by default)
  throwingNew[-]        assume operator new throws on failure (off by default)
  referenceBinding[-]   a temporary will not bind to an non-const
                        lvalue reference (off by default)
/ZH:SHA_256             use SHA256 for file checksum in debug info (experimental)

/Zo[-] generate richer debugging information for optimized code (on by default)
/ZW enable WinRT language extensions
/constexpr:depth<N>     use <N> as the recursion depth limit
                        for constexpr (default: 512)
/constexpr:backtrace<N> show <N> constexpr evaluations
                        in diagnostics (default: 10)
/constexpr:steps<N>     terminate constexpr evaluation after
                        <N> steps (default: 100000)
/ZI enable Edit and Continue debug info
/openmp enable OpenMP 2.0 language extensions

About exactly in the middle of the above my eyes caught on this little gem…

Code: [Select]
sizedDealloc[-]       enable C++14 global sized deallocation functions (on by default)

Now that really sounds interesting!  There’s mention of a 2nd parameter somehow specifying the size of the memory to be released!  Definitely worth a try!  And it worked!  After like four days struggling with this it finally worked like a charm!

As I had mentioned I had been doing internet searches on various terms found within the linker error messages but hadn’t found any mention of this /Zc:sizedDealloc- thing.  But when I did a search on sizedDealloc the top two returned results were these…

About 376 results

Issue 1309263004: Add /Zc:sizedDealloc- to work around VS 2015 ...

https://codereview.chromium.org/1309263004

Aug 31, 2015 ... Add /Zc:sizedDealloc- to work around VS 2015 bug VS 2015's support for VC++ 14 sized deallocs causes __global_delete to be exported from ...
BUILD 2015 News: Visual Studio Code, Visual Studio 2015 RC ...
https://blogs.msdn.microsoft.com/.../build-2015-news-visual-studio-code-visual-studio-2015-rc-team-foundation-server-2015-rc-visual-studio...

Apr 29, 2015 ... For now using the compiler switch /Zc:sizedDealloc- will workaround the issue ... In the meantime the /Zc option seems to workaround the issue.

Navigating to the top one revealed this...
Issue 1309263004: Add /Zc:sizedDealloc- to work around VS 2015 bug (Closed)

Description

Add /Zc:sizedDealloc- to work around VS 2015 bug VS 2015's support for VC++ 14 sized deallocs causes __global_delete to be exported from many object files, causing obscure and hard to track linker errors. The best workaround, as suggested here: https://connect.microsoft.com/VisualStudio/feedback/details/1379741/-global-delete-function-is-inconsistently-generated Is to use /Zc:sizedDealloc- to disable the new C++14 feature support that is triggering this behavior change.

And navigating to the Microsoft link revealed this where Microsoft acknowledged it as a bug...
Cary Sandvig

I'm running into the 'C2956: sized deallocation …' placement new/delete problem that bill pittore posted. Has this been officially acknowledged? Is there a work-around?

1.   10 months ago

Ayman Shoukry [MSFT]
@bill pittore  & Cary Sandvig

Thanks for reporting the issue! Yes, this is a bug that we will target fixing before the RTM (final) release. For now using the compiler switch /Zc:sizedDealloc- will workaround the issue till we fix it.

Thanks!

So I was really amazed by all that!  I thought it was just my clinging to older software that was causing the problem.  But others were clearly running into the problem which was in fact a bug!  And I was at least right in assuming the problem was related to changes made to the compiler to support the new C++ features.

Apparently Microsoft never fixed it or at least the software I downloaded three weeks ago when I bought a new HP Envy laptop didn’t reflect the fix.  But its behind me now and I can move on.  But the whole affair didn’t do a d*** thing to improve my opinion of new  bleeding edge software.  Here’s Test1.cpp again but where you’ll see three different command line compilation strings at top…

Code: [Select]
// cl Test1.cpp Strings.cpp /O1 /Os /GS- /Zc:sizedDealloc- /link TCLib.lib kernel32.lib   << For TCLib
// cl Test1.cpp Strings.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib                     << Won't Work With TCLib.lib
// cl Test1.cpp Strings.cpp /O1 /Os /D "_CRT_SECURE_NO_WARNINGS"                          << For Standard VC Without TCLib.lib
#define UNICODE
#define _UNICODE      // 122,880 bytes
#include <windows.h>  //   4,608 bytes
#include "stdio.h"    // =============   
#include "tchar.h"    // 118,272 bytes     26.66 times smaller
#include "Strings.h"

int _tmain()
{
 int iParseCount=0;
 String* pStrs=NULL;
 String s1;
 
 s1=_T("Zero, One, Two, Three, Four, Five");
 s1.Print(_T("s1 = "),true);
 iParseCount=s1.ParseCount(_T(','));
 _tprintf(_T("iParseCount = %d\n\n"),iParseCount);
 pStrs=new String[iParseCount];
 s1.Parse(pStrs, _T(','), iParseCount);
 for(int i=0; i<iParseCount; i++)
 {
     pStrs[i].LTrim();
     pStrs[i].Print(true);
 }
 delete [] pStrs;
 getchar();
 
 return 0;
}

Using the top one with the /Vc:sizedDealloc- switch produces this result on the command line…

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test9>cl Test1.cpp Strings.cpp /O1 /Os /GS- /Zc:sizedDealloc- /link TCLib.lib kernel32.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

Test1.cpp
Strings.cpp
Generating Code...
Microsoft (R) Incremental Linker Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:Test1.exe
TCLib.lib
kernel32.lib
Test1.obj
Strings.obj

C:\Code\VStudio\VC15\LibCTiny\x64\Test9>Test1.exe
s1 = Zero, One, Two, Three, Four, Five
iParseCount = 6

Zero
One
Two
Three
Four
Five

You can see in there I also executed the program – Test1.exe, right after the successful compile.  It works just fine and you can see the results are the same as from the PowerBASIC Console Compiler program.  Its coming in 4,608 bytes, which is a lot smaller even than the PowerBASIC one.  The 2nd command line compilation string at top of Test1.cpp is the one without /Zc:sizedDealloc- switch which won’t link and gives the errors I’ve described.  The 3rd command line compilation string is the simplest…

Code: [Select]
cl Test1.cpp Strings.cpp /O1 /Os /D "_CRT_SECURE_NO_WARNINGS"                           

…and just uses the typical linkage with the C Standard Library through LIBCMT.LIB or whatever they changed it to as a result of their new UCRT Refactoring.  That one is coming in 122,880 bytes.  If you want to test compile/run that make sure to change the …

Code: [Select]
“stdio.h”   to  <stdio.h> or <cstdio>
“string.h” to <string.h>
“tchar.h”  to <tchar.h>

…for the reasons I previously described.  At the top of Test1.cpp I have it all compared in terms of executable program sizes…

Code: [Select]
#define _UNICODE      // 122,880 bytes
#include <windows.h>  //   4,608 bytes
#include "stdio.h"    // =============   
#include "tchar.h"    // 118,272 bytes     26.66 times smaller

So using my TCLib.lib we’ve removed 118,272 bytes and made it 26.66 times smaller!

Considering the difficulties I had in getting my object delete calls to compile/link, it might behoove us to check that memory isn’t leaking.   I knew it shouldn’t be on this code, which I’ve been using since like forever, and had tested it countless times, but I really didn’t trust VC19.

If you look in Strings.h near the top is a #define debug statement commented out.  Uncomment that to produce debug output.  Lets run Test1.cpp after recompiling it to make sure all the memory release calls are being made.   Here is the String::Parse() method…

Code: [Select]
void String::Parse(String* pStr, TCHAR delimiter, size_t iParseCount)
{
 TCHAR* pBuffer=new TCHAR[this->iLen+1];    // Allocate copy buffer large enough to contain whole String
 if(pBuffer)                                // pointed to by this->lpBuffer.  Then declare a TCHAR* p to
 {                                          // point to the beginning of that copy buffer.  Then declare
    TCHAR* p=pBuffer;                       // another TCHAR* to point to the beginning of this->lpBuffer.
    TCHAR* c=this->lpBuffer;                // Using a while loop, iterate through this->lpBuffer one
    while(*c)                               // TCHAR at a time comparing the character read out of
    {                                       // this->lpBuffer to the delimiter parameter.  If the TCHAR is
       if(*c==delimiter)                    // not a delimiter, then simply copy it to pBuffer.  If it is
          *p=0;                             // a delimiter, write a NULL TCHAR to pBuffer.  The end result
       else                                 // of this copy process will be a pBuffer with NULLs where the
          *p=*c;                            // delimiters were in this->lpBuffer.  Then simply run through
       p++, c++;                            // pBuffer with a for loop collecting our NULL terminated
    }                                       // Strings.  Note I reset p used in the while loop back to the
    *p=0, p=pBuffer;                        // beginning of pBuffer to start collecting Strings from the
    for(size_t i=0; i<iParseCount; i++)     // beginning.  Further note that this line...
    {                                       //
        pStr[i]=p;                          //                        pStr[i]=p;
        p=p+pStr[i].iLen+1;                 //
    }                                       // ...will trigger an operator= call because it represents a
    delete [] pBuffer;                      // NULL terminated TCHAR string being assigned to a String
 }                                          // object.
}

And here is console output from my debug output run and I have comments to the right explaining what’s happening..

Code: [Select]
C:\Code\VStudio\VC15\LibCTiny\x64\Test9>Test1.exe

Entering String Uninitialized Constructor!                 // This output is a call to the uninitialized String
  this = 0000008FE10FFB80                                  // Constructor caused by ‘String s1;’ in main().
Leaving String Uninitialized Constructor!

Entering String::operator=(TCHAR*)                         // This is an operator= call caused by the assignment of
  this            = 0000008FE10FFB80                       // "Zero, One, Two, Three, Four, Five" to s1 at the top of
  pStr            = Zero, One, Two, Three, Four, Five      // main().  Note that the original memory allocation for
  _tcslen(pStr)   = 33                                     // s1 only provided for a String which could hold 15 TCHARS.
  this->iCapacity = 15                                     // The logic within operator= determines that the string
  Had To Reallocate!                                       // needing to be assigned is too big to copy to the original
  this->iCapacity = 79                                     // 16 TCHAR buffer, so the original buffer is Released and
  this->lpBuffer  = Zero, One, Two, Three, Four, Five      // a new and larger one allocated by my somewhat aggressive
Leaving String::operator=(TCHAR*)                          // memory expansion logic.

s1 = Zero, One, Two, Three, Four, Five                     // This line of output is from an s1.Print(1) call.
iParseCount = 6                                            // And this one shows the result of calling String::ParseCount.

Entering String Uninitialized Constructor!                 // Now this interesting output is from this line…             
  this = 000002822AB13D78                                  //
Leaving String Uninitialized Constructor!                  //             pStrs=new String[iParseCount];
                                                           //
Entering String Uninitialized Constructor!                 // …which is a dynamic memory allocation call for six String
  this = 000002822AB13D90                                  // objects to be created on the heap.  They will be empty
Leaving String Uninitialized Constructor!                  // String objects, so that will trigger calls to the
                                                           // uninitialized constructor for the String Class.
Entering String Uninitialized Constructor!
  this = 000002822AB13DA8
Leaving String Uninitialized Constructor!

Entering String Uninitialized Constructor!
  this = 000002822AB13DC0
Leaving String Uninitialized Constructor!

Entering String Uninitialized Constructor!
  this = 000002822AB13DD8
Leaving String Uninitialized Constructor!

Entering String Uninitialized Constructor!
  this = 000002822AB13DF0
Leaving String Uninitialized Constructor!

Entering String::operator=(TCHAR*)                          // Here begin the calls to operator= near the end of
  this            = 000002822AB13D78                        // String::Parse() which are triggered by running through
  pStr            = Zero                                    // pBuffer collecting the NULL terminated String Objects.
  _tcslen(pStr)   = 4                                       // Each pStrs[i] represents an uninitialized String which
  this->iCapacity = 15                                      // is now being set to a TCHAR NULL terminated String.
  this->lpBuffer  = Zero
Leaving String::operator=(TCHAR*)

Entering String::operator=(TCHAR*)
  this            = 000002822AB13D90
  pStr            =  One
  _tcslen(pStr)   = 4
  this->iCapacity = 15
  this->lpBuffer  =  One
Leaving String::operator=(TCHAR*)

Entering String::operator=(TCHAR*)
  this            = 000002822AB13DA8
  pStr            =  Two
  _tcslen(pStr)   = 4
  this->iCapacity = 15
  this->lpBuffer  =  Two
Leaving String::operator=(TCHAR*)

Entering String::operator=(TCHAR*)
  this            = 000002822AB13DC0
  pStr            =  Three
  _tcslen(pStr)   = 6
  this->iCapacity = 15
  this->lpBuffer  =  Three
Leaving String::operator=(TCHAR*)

Entering String::operator=(TCHAR*)
  this            = 000002822AB13DD8
  pStr            =  Four
  _tcslen(pStr)   = 5
  this->iCapacity = 15
  this->lpBuffer  =  Four
Leaving String::operator=(TCHAR*)

Entering String::operator=(TCHAR*)
  this            = 000002822AB13DF0
  pStr            =  Five
  _tcslen(pStr)   = 5
  this->iCapacity = 15
  this->lpBuffer  =  Five
Leaving String::operator=(TCHAR*)

Zero
One
Two
Three
Four
Five

Entering String Destructor!      // The Strings will be destroyed in the reverse order from which they were created.
  this = 000002822AB13DF0        // There are seven Strings to be deallocated and we have seven Destructor() calls.
Leaving String Destructor!       // Everything is in order!

Entering String Destructor!
  this = 000002822AB13DD8
Leaving String Destructor!

Entering String Destructor!
  this = 000002822AB13DC0
Leaving String Destructor!

Entering String Destructor!
  this = 000002822AB13DA8
Leaving String Destructor!

Entering String Destructor!
  this = 000002822AB13D90
Leaving String Destructor!

Entering String Destructor!
  this = 000002822AB13D78
Leaving String Destructor!

Entering String Destructor!
  this = 0000008FE10FFB80
Leaving String Destructor!

So basically, everything is good.  Bear with me for a couple minutes and we’ll look at a C version of the above, and I think you’ll agree the C++ version is better.

Code: [Select]
// cl Parse.cpp /O1 /Os /D "_CRT_SECURE_NO_WARNINGS"
// cl Parse.cpp /O1 /Os /GS- /link TCLib.lib kernel32.lib
#ifndef UNICODE
   #define  UNICODE
#endif
#ifndef _UNICODE   
   #define  _UNICODE
#endif   
#include <windows.h>
#include "cstdlib"
#include "stdio.h"      // 122,368 Bytes With Microsoft Libraries
#include "string.h"     //   4,608 Bytes With TCLib.lib


size_t iParseCount(const wchar_t* pString, wchar_t cDelimiter)
{
 int iCtr=0;       //reflects # of strings delimited by delimiter.
 const wchar_t* p;                     

 p=pString;
 while(*p)
 {
   if(*p==cDelimiter)
      iCtr++;
   p++;
 }

 return ++iCtr;
}


void Parse(wchar_t** pStrings, const wchar_t* pDelimitedString, wchar_t cDelimiter, int iParseCount)
{
 wchar_t* pBuffer=NULL;             // In Parse() we receive a wchar_t** (pointer to pointer)
 const wchar_t* c;                  // buffer large enough to hold pointers to iParseCount
 wchar_t* p;                        // number of strings delimited by cDelimiter.  The logic
 size_t i=0;                        // used here is to create a secondary buffer as large as
                                    // the string passed in the 2nd parameter, i.e.,
 pBuffer=(wchar_t*)malloc           // pDelimiterString, then work through the pDelimitedString
 (                                  // buffer with a while loop copying characters from
  wcslen(pDelimitedString)*2+2      // pDelimitedString to our new buffer - pBuffer, one
 );                                 // character at a time.  When a delimiter is hit as
 if(pBuffer)                        // specified by cDelimiter, place a null byte in pBuffer
 {                                  // instead of the delimiter so as to null terminate that
    pBuffer[0]=0;                   // string.  The end result of this process of running through
    p=pBuffer;                      // pDelimitedString and substituting nulls in place of
    c=pDelimitedString;             // the delimiter in pBuffer is to create an array of null
    while(*c)                       // terminated strings in pBuffer.  The final part of the
    {                               // algorithm simply needs to collect all those strings
       if(*c==cDelimiter)           // from pBuffer, and place pointers to them in our wchar_t**
          *p=0;                     // pStrings buffer.  Of course, an individual memory alloc-
       else                         // ation with malloc will be needed for each delimited string.
          *p=*c;                    // Then the temporary pBuffer must be released here.  All the
       p++, c++;                    // memory allocations for everything will have to be unwound
    }                               // in main().
    *p=0, p=pBuffer;
    for(int i=0; i<iParseCount; i++)
    {
       pStrings[i]=(wchar_t*)malloc(wcslen(p)*2+2);
       wcscpy(pStrings[i],p);
       p=p+wcslen(pStrings[i])+2;       
    }
    free(pBuffer);
 }
}


void LTrim(wchar_t* pStr)
{
 size_t iCt=0, iLen=0;

 iLen=wcslen(pStr);
 for(size_t i=0; i<iLen; i++)
 {
     if(pStr[i]==32 || pStr[i]==9)
        iCt++;
     else
        break;
 }
 if(iCt)
 {
    for(size_t i=iCt; i<=iLen; i++)
        pStr[i-iCt]=pStr[i];
 }
}


int main()
{
 wchar_t szString[]=L"Zero, One, Two, Three, Four, Five";
 wchar_t** pStrs=NULL;
 size_t iStrings;

 iStrings=iParseCount(szString, L',');                      // First have to allocate pStrs buffer, which is a pointer
 pStrs=(wchar_t**)malloc(iStrings*sizeof(wchar_t*));        // to pointer buffer (wchar_t**) to hold six wchar_t* pointers
 if(pStrs)                                                  // to the individual strings to be parsed, i.e., "Zero", "One",
 {                                                          // etc.  Then call Parse() function passing in the pStrs **
    Parse(pStrs,szString,L',',iStrings);                    // buffer, the delimiter character, and the result of ParseCount.
    for(size_t i=0;i<iStrings; i++)                         // After Parse() successfully terminates, display trimmed strings.
    {                                                       // Note that the Parse() function receives the pStr wchar_t**
        LTrim(pStrs[i]);                                    // buffer, but must make individual memory allocations for the
        wprintf(L"%s\t%Iu\n",pStrs[i],wcslen(pStrs[i]));    // six strings, and place those pointers in that 24/48 byte
    }                                                       // buffer.  Here, back in main(), we'll have to release the
    for(size_t i=0;i<iStrings; i++)                         // memory for the six string pointers, then the memory for the
        free(pStrs[i]);                                     // pStrs wchar_t** buffer.  In C++ with Parse() as a member of
    free(pStrs);                                            // my String Class, a simple delete [] pStrs call will release
 }                                                          // the pointer to pointer buffer, but first call destructors on
 getchar();                                                 // the individual Strings.  So memory release is all done in
                                                            // one shot.
 return 0;
}

And interestingly, its coming in 4,608 bytes exactly the same as the C++ version!  But where its nastier than the C++ version is that one must first declare a …

Code: [Select]
wchar_t** pStrs;

…because first the buffer must be allocated to hold six pointers, which will be the pointers to each string once Parse() parses them.  And it’ll have to place them in that buffer just like the C++ program with the String Class does.  It gets a little nastier in Parse() too.  Couple more lines of code it took.  But the real nasty part comes at the end of main where free() calls must be made on each of the strings allocated in Parse(), followed with a free() for the wchar_t** pointer to pointer buffer. 

In a C++ program one uses delete like so to de-allocate an array of objects…

Code: [Select]
delete [] pStrs;


That differs from releasing just one object, which is this…

Code: [Select]
delete pStr;

If one were to forget the ‘[]’ symbols when deleting an array, what would happen is that all the memory would be released ultimately with free, but only the first object in the array would have its destructor called.  So any memory allocated within any of the objects beyond the first one would leak.  If you look at the debug output shown above for the String Class C++ example you’ll see that destructors were called on every object instantiated.  That’s about all for now.  My last post on this will come shortly when I’ll present my String Class which can be used in ASCI or UNICODE.

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
And attached is a zip file containing everything noteworthy in my last posts, including a TCLib.lib file supporting ansi/unicode as developed with all the files I presented.

Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
I've got my String Class converted back to all the TCHAR macros, and am in the process of testing all x/86 / x64 /UNICODE /ANSI combinations.  So far no bugs or problems.  Had a few minutes to kill today and did an internet search on algorithms to convert binary floating point numbers to asci.  Of course, that's the big weakness as I see it of what I have so far.  I consider it detestable that I have to LoadLibrary/GetProcAddress on msvcrt.dll to get floats converted when I've done away with that large footprint library except for that.  And I found a real easy algorithm to do it.  I think I can work it in.  If I can, my work on it will be finished.  Here's the little example algorithm with some debug printf statements.  Its working on converting 1234.5678, which I entered with scanf...

Code: [Select]
// cl float03.cpp
#include<stdio.h>

void flot(char* p, double x)
{
 int n,i=0,k=0;

 printf("Entering flot()\n");
 printf("  x = %16.8f\n",x);
 n=(int)x;
 printf("  n = %d\n\n",n);
 printf("  i\tn\tx\n");
 printf("  =======================\n");
 while(n>0)
 {
   x=x/10;
   n=(int)x;
   i++;
   printf("  %d\t%d\t%12.4f\n",i, n, x);
 }

 printf("\n");
 *(p+i) = '.';
 x=x*10;
 n = (int)x;
 printf("  x = %f\n",x);
 printf("  n = %d\n",n);
 x = x-n;
 printf("  x = %f\n\n",x);

 while(n>0)
 {
   if(k == i)
      k++;
   *(p+k)=48+n;
   x=x*10;
   n = (int)x;
   x = x-n;
   k++;
   printf("  %d\t%d\t%f\t%s\n",k,n,x,p);
 }
 *(p+k) = '\0';
 printf("Leaving flot()\n\n");
}

int main()
{
 char a[32]={0};
 double x;
 
 char* p=&a[0];
 printf("Enter the float value...");
 scanf("%lf",&x);
 flot(p,x);
 printf("The value=%s\n",p);
 getchar();

 return 0;
}

/*
C:\Code\VStudio\VC15\LibCTiny\x64\Test9>float03.exe
Enter the float value...1234.5678
Entering flot()
  x =    1234.56780000
  n = 1234

  i     n       x
  =======================
  1     123         123.4568
  2     12           12.3457
  3     1             1.2346
  4     0             0.1235

  x = 1.234568
  n = 1
  x = 0.234568

  1     2       0.345678        1
  2     3       0.456780        12
  3     4       0.567800        123
  4     5       0.678000        1234.
  6     6       0.780000        1234.5
  7     7       0.800000        1234.56
  8     8       0.000000        1234.567
  9     0       0.000000        1234.5678
Leaving flot()

The value=1234.5678
*/

I'm hoping to have luck with that.
« Last Edit: March 02, 2016, 03:22:29 AM by Frederick J. Harris »

Offline James C. Fuller

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 595
  • User-Rate: +11/-8
Fred,
  Here is my bc9Basic translation of your demo using your TCLib.
  I use a unicode lexer "ULEX" to create a unicode version of the demo.
  I added two items to your tchar.h file to make it more compatible with bc9Basic translations.

James

==============================================================================

'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
'Test of Fred's Minimal c++ TCLib.lib using bc9Basic
'added to Fred's "tchar.h"
'typedef wchar_t     _TCHAR;
'#define _gettchar    getchar
'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
$CPP
$NOMAIN
$ONEXIT "ULEX.EXE $FILE$.CPP TCHARXLATER_VC.TXT"
$ONEXIT "FRED.BAT $FILE$"

'==============================================================================
$HEADER
    #include <windows.h>
    #include "stdio.h"
    #include "tchar.h"
    #include "Strings.h"
    typedef String fstring;
$HEADER
'==============================================================================
Function main()
    Local As int iParseCount
    Local As fstring Ptr pStrs
    Raw As fstring s1
    s1 = ("Zero, One, Two, Three, Four, Five")
    s1.Print("s1 = ",_true)
    iParseCount=s1.ParseCount(_T("','"E))
    Print "iParseCount = ",iParseCount
    pStrs = new fstring[iParseCount]
    s1.Parse(pStrs,("','"E),iParseCount)
    xFor int i = 0 While i < iParseCount By i++
        pStrs[i].LTrim()
        pStrs[i].Print(_true)
    xNext
    delete [] pStrs
    getchar()
    Function = 0
End Function



Batch File:
It would be easier if String was precompiled and stuck in a library directory but this was just a first test.
Code: [Select]
@setlocal enableextensions enabledelayedexpansion
@ECHO OFF

SET F=%~nx1
IF EXIST "%F%.cpp" (
  SET FN="%F%.cpp"
  GOTO start
)
GOTO usage

:start
SET XTYPE=x86_amd64
CALL "%VS140COMNTOOLS%..\..\VC\vcvarsall.bat" %XTYPE%

cl %FN% Strings.cpp /O1 /Os /GS- /Zc:sizedDealloc- /link TCLib.lib kernel32.lib
:cleanup
ECHO Finished!
IF EXIST "%F%.obj" del "%F%.obj"
IF EXIST "Strings.obj" del "Strings.obj"

GOTO done

:usage
ECHO **************************************************************
ECHO  Usage:  FRED.BAT MainFile no extension cpp assumed
ECHO **************************************************************
:done

C++ source
Code: [Select]
// *********************************************************************
//  Created with bc9Basic - BASIC To C/C++ Translator (V) 9.2.4.5 (2016/03/01)
//       The bc9Basic translator (bc9.exe) was compiled with
//                           g++ (tdm64-1) 5.1.0
// ----------------------------------------------------------------------
//                 BCX (c) 1999 - 2009 by Kevin Diggins
// *********************************************************************
//              Translated for compiling with a C++ Compiler
//                           On MS Windows
// *********************************************************************
#ifndef UNICODE
#define UNICODE
#endif
#ifndef _UNICODE
#define _UNICODE
#endif
#define _X(y) y

// ***************************************************
// Compiler Defines
// ***************************************************
#ifndef __cplusplus
#error A C++ compiler is required
#endif
// $HEADER BEGIN
#include <windows.h>
#include "stdio.h"
#include "tchar.h"
#include "Strings.h"
typedef String fstring;
// $HEADER END


// *************************************************
//        User's GLOBAL ENUM blocks
// *************************************************

// *************************************************
//            System Defined Constants
// *************************************************

typedef const _TCHAR* ccptr;
#define CCPTR const _TCHAR*
#define cfree free
#define WAITKEY _tsystem(_T("pause"))
#define EQU ==
#define NOT_USED(x) if(x);
#define CTLHNDL(id) GetDlgItem(hWnd,id)
_TCHAR   *g_cptr_;  // dummy var for not used returns
size_t  g_dum1_;  // dummy var for not used returns
#define cSizeOfDefaultString 2048

// *************************************************
//            User Defined Constants
// *************************************************

// *************************************************
//          User Defined Types, Unions and Classes
// *************************************************


// *************************************************
//            User Global Variables
// *************************************************

static _TCHAR*   *g_argv;
static int     g_argc;


// *************************************************
//               User Prototypes
// *************************************************

int     _tmain (int, _TCHAR**);

// *************************************************
//            User Global Initialized Arrays
// *************************************************



// *************************************************
//                 Runtime Functions
// *************************************************


// *************************************************
//       User Subs, Functions and Class Methods
// *************************************************

int _tmain (int argc, _TCHAR** argv)
{
    g_argc = argc;
    g_argv = argv;
    int      iParseCount = {0};
    fstring*  pStrs = {0};
    fstring  s1;
    s1 = (_T("Zero, One, Two, Three, Four, Five"));
    s1.Print(_T("s1 = "), true);
    iParseCount = s1.ParseCount(_T(','));
    _tprintf(_T("%ls%d\n"), _T("iParseCount = "), (int)iParseCount);
    pStrs = new  fstring[iParseCount];
    s1.Parse(pStrs, (','), iParseCount);
    {
        int      i;
        for(i = 0; i < iParseCount; i++)
        {
            pStrs[i].LTrim();
            pStrs[i].Print( true);
        }
    }

    delete[] pStrs;
    _gettchar();
    return 0;
}





Offline Frederick J. Harris

  • Hero Member
  • *****
  • Posts: 914
  • User-Rate: +16/-0
    • Frederick J. Harris
That's pretty amazing James.  I see in the top code you posted you have an #include "Strings.h".  Did you have to translate that to Bcx9 to get it to work?  Obviously, for the C++ translated code it could be used as is.  Hmmm.  Got me to thinking....

Maybe I’m not fully understanding what you’ve done.  Do you need to translate this….

Code: [Select]
'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
'Test of Fred's Minimal c++ TCLib.lib using bc9Basic
'added to Fred's "tchar.h"
'typedef wchar_t     _TCHAR;
'#define _gettchar    getchar
'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
$CPP
$NOMAIN
$ONEXIT "ULEX.EXE $FILE$.CPP TCHARXLATER_VC.TXT"
$ONEXIT "FRED.BAT $FILE$"

'==============================================================================
$HEADER
    #include <windows.h>
    #include "stdio.h"
    #include "tchar.h"
    #include "Strings.h"
    typedef String fstring;
$HEADER
'==============================================================================
Function main()
    Local As int iParseCount
    Local As fstring Ptr pStrs
    Raw As fstring s1
    s1 = ("Zero, One, Two, Three, Four, Five")
    s1.Print("s1 = ",_true)
    iParseCount=s1.ParseCount(_T("','"E))
    Print "iParseCount = ",iParseCount
    pStrs = new fstring[iParseCount]
    s1.Parse(pStrs,("','"E),iParseCount)
    xFor int i = 0 While i < iParseCount By i++
        pStrs[i].LTrim()
        pStrs[i].Print(_true)
    xNext
    delete [] pStrs
    getchar()
    Function = 0
End Function

…to a C++ executable in order to run it?  In other words, is C++ the compiler for bc9basic?  Or is there one option for running the bc9basic in its ‘native’ mode, and another ‘modality’ to convert it to C++ bninary code?  I guess that's what I'm asking.  If bc9basic doesn't run on its own bc9basic to binary converter, then translating Strings.h to bc9basic doesn't make sense.

You may recall a couple years ago Daniel Corbier made a plea here for help in writing a PowerBAIC to C++ converter.  I never contacted him about it or responded in any way simply because I thought it would be so difficult to do.  He seemed to have a background in working on such projects (and I didn't) so I thought he may have a chance of pulling it off.

I did what I did, i.e., translating all my PowerBASIC code slowly and laborously to C++ one line at a time, simply because I knew with 100% certainty I would succeed, short of dying or something.  But it looks like you’ve come close with your Bc9basic project.   
« Last Edit: March 05, 2016, 01:33:43 AM by Frederick J. Harris »

Offline James C. Fuller

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 595
  • User-Rate: +11/-8
Fred,
  bc9Basic is just a translator. Basic syntax source -> c or c++ source. You supply your own c/c++ compiler.
c/c++ source produced using bc9Basic can be compiled with supplied batch files for PellesC (32/64), TinyC(32/64), Visual Studio(32/64),TDM-GCC(32.64),Nuwen(64), and any of the other MinGW flavors.
The basic source can also include any c/c++ source code, although I try to use basic syntax whenever possible when coding.
It is a direct descendant of BCX, written by Kevin Diggins, which has been around since the dos days; the original written in PbDos.

One of it's nice features is the $ONEXIT directive which allows you to run external programs or batch files after the translation is complete. There can be more than one and are run in sequence. By using $ONEXIT I can test a number of different compilers with just an unremming of code.

'For TDM-GCC
'first run the unicode lexer on translation
'$ONEXIT "ULEX.EXE $FILE$.CPP TCHARXLATER.TXT"
'now run the batch file to compile
'$ONEXIT "TDMGPP.BAT $FILE$ -m64 con"

'Visual Studio
'run the unicode lexer on translation
'$ONEXIT "ULEX.EXE $FILE$.CPP TCHARXLATER_VC.TXT"
'run the batch file to compile
$ONEXIT "VSCPP.BAT $FILE$ -m64 con"
 
Any code between $HEADER is written to the top of the c/c++ source as is.

I did a special batch file for this exercises where I used your command line compile so your String class gets compiled and linked.

The flexibility of bc9Basic gives many options to attack a problem.

James
« Last Edit: March 05, 2016, 11:27:41 AM by James C. Fuller »