Theo's Forum

IT-Consultant: Charles Pegge => OxygenBasic => Topic started by: Charles Pegge on April 23, 2008, 05:25:20 PM

Title: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Charles Pegge on April 23, 2008, 05:25:20 PM

This is part of the O2 project.

o2linux.zip: (Posted 26 April 2008 )

Containg liboxygen.so and demo: OxygenTestFBLD which uses run-time linking to liboxygen. Both files reside in the same directory. liboxygen.so does not have to be installed in /usr/lib.

Oxygen.zip: 
Updated 30 April 2008 with soft coded instruction set extensions (also corrections to 3DNOW set)
Updated 25 April 2008 with o2_buf function


Containing Oxygen.dll provides dynamic assembly services. Pass a source code string. It will returned the assembled binary in a relocatable string. It can be used with both Powerbasic and FreeBasic. There will also be a Linux version soon. It is the low level counterpart of the thinBasic_Oxygen module.

http://community.thinbasic.com/index.php?topic=1637.0


FBGFX_t.zip

This is a graphic demo, showing how the assembler can be used with a FreeBasic program. Thanks to Petr Schreiber for the Plasma graphics. We have several demos of this in thinBasic.

Both zips contain source code and executables.

header for PB

Code: [Select]

    'PB interface
     MACRO zstring=ASCIIZ
     DECLARE FUNCTION o2_asm       LIB "oxygen.dll" ALIAS "o2_asm"       (zstring) AS STRING   ' assemble
     DECLARE FUNCTION o2_link      LIB "oxygen.dll" ALIAS "o2_link"      (zstring) AS STRING   ' o2 link **
     DECLARE FUNCTION o2_view      LIB "oxygen.dll" ALIAS "o2_view"      (zstring) AS STRING   ' o2 coding
     DECLARE FUNCTION o2_asm_file  LIB "oxygen.dll" ALIAS "o2_asm_file"  (zstring) AS STRING   ' assemble from file
     DECLARE FUNCTION o2_view_file LIB "oxygen.dll" ALIAS "o2_view_file" (zstring) AS STRING   ' o2 coding from file
     DECLARE FUNCTION o2_error     LIB "oxygen.dll" ALIAS "o2_error"     ()        AS STRING   ' error message if any
     DECLARE FUNCTION o2_len       LIB "oxygen.dll" ALIAS "o2_len"       ()        AS LONG     ' length of returned string
     DECLARE FUNCTION o2_buf       LIB "oxygen.dll" ALIAS "o2_buf"       (BYVAL LONG)AS LONG   ' buffer selector 0..511 (returns pointer)
     ' ** o2_link is automatically called by o2_asm when the assembly is successful.

Header for FB with Microsoft

Code: [Select]

Extern "Windows-MS"
     DECLARE FUNCTION o2_asm       LIB "oxygen.dll" ALIAS "o2_asm"       (byval as zstring ptr) AS long ' assemble
     DECLARE FUNCTION o2_link      LIB "oxygen.dll" ALIAS "o2_link"      (byval as zstring ptr) AS long ' o2 link **
     DECLARE FUNCTION o2_view      LIB "oxygen.dll" ALIAS "o2_view"      (byval as zstring ptr) AS long ' o2 coding
     DECLARE FUNCTION o2_asm_file  LIB "oxygen.dll" ALIAS "o2_asm_file"  (byval as zstring ptr) AS long ' assemble from file
     DECLARE FUNCTION o2_view_file LIB "oxygen.dll" ALIAS "o2_view_file" (byval as zstring ptr) AS long ' o2 coding from file
     DECLARE FUNCTION o2_error     LIB "oxygen.dll" ALIAS "o2_error"     ()                     AS long ' error message if any
     DECLARE FUNCTION o2_len       LIB "oxygen.dll" ALIAS "o2_len"       ()                     AS long ' length of returned string
     DECLARE FUNCTION o2_buf       LIB "oxygen.dll" ALIAS "o2_buf"       (byval as long)        AS long '  buffer selector 0..511 (returns pointer)
End Extern

Header for FB with Linux (using runtime linking)

Code: [Select]

     dim shared pl as any ptr
     pl = dylibload ("liboxygen.so")
     if pl=0 then print "Library not located":end
     dim shared o2_asm       as FUNCTION (byval as zstring ptr) AS long ' assemble
     dim shared o2_link      as FUNCTION (byval as zstring ptr) AS long ' o2 link **
     dim shared o2_view      as FUNCTION (byval as zstring ptr) AS long ' o2 coding
     dim shared o2_asm_file  as FUNCTION (byval as zstring ptr) AS long ' assemble from file
     dim shared o2_view_file as FUNCTION (byval as zstring ptr) AS long ' o2 coding from file
     dim shared o2_error     as FUNCTION ()                     AS long ' error message if any
     dim shared o2_len       as FUNCTION ()                     AS long ' length of returned string
     dim shared o2_buf       as FUNCTION (byval as long)        AS long ' buffer selector 0..511 (returns pointer)

     o2_link      = dylibsymbol(pl,"o2_link")
     o2_asm       = dylibsymbol(pl,"o2_asm")
     o2_asm_file  = dylibsymbol(pl,"o2_asm_file")
     o2_view      = dylibsymbol(pl,"o2_view")
     o2_view_file = dylibsymbol(pl,"o2_view_file")
     o2_error     = dylibsymbol(pl,"o2_error")
     o2_len       = dylibsymbol(pl,"o2_len")
     o2_buf       = dylibsymbol(pl,"o2_buf")

     ' ** o2_link is automatically called by o2_asm when the assembly is successful.


Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Petr Schreiber on April 23, 2008, 06:16:55 PM

Thanks Charles,

but the original thinbasic plasma program is by "misthema", I just did the doubtful assembler translation :)
I must say using ( breathing ) the Oxygen is very comfortable, and brought my attention to assembler after long time.

Petr

Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Theo Gottwald on April 23, 2008, 07:14:47 PM

Looks interesting.
On the logn run, I hope to see more examples for PB because such a system could have a future.

I doubt, that any implementation can have all ASM Codes because this is a too dynamic area which changes all 6 Month.
In such a way, a Freeware DLL can keep PB up-to-date for all the Mnemonics.

While I think the ASM-Codes will be in separate Text files, because its just easier to edit then in Strings.
Therefore maybe there could be DLL-CAlls which make it even more easy to directly make all these things.

Example:

Code: [Select]

' Generate complete PB-Usable Binary code from textfile
OASM "filespec.txt",ASM$
ORUN ASM$(,Debug-Options)

Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Charles Pegge on April 23, 2008, 09:39:02 PM

Hi Theo,

Below is the example interface for PB. I wanted to keep the strings as generic as possible, so source code and file specs are passed to Oxygen in zstrings/asciiz. Oxygen then returns a pointer to a buffer containing the result string. O2_len() returns the length of this string. This method will work on any platform, since it does not assume a BSTR. This inferface can be wrapped to make it look simpler but this example shows the raw details which are not too burdensome.

I agree about new codes being frequently developed. We could provide a soft-coding for all the extension instructions - a small datafile that would allow new instructions to be added. The 'grammar' of the recent extensions is simpler than that of the base instruction set, so that would be quite easy, and may help to improve the performance of the assembler.

My top priority at the moment is to continue refining the test program - about 1200 checks so far - then it will be safe to beef up the program.

Code: [Select]

' OxygenTestPB.bas
' Oxygen Assembler 'Asmosphere' project
' Charles E V Pegge
' 19 April 2008

    'PB interface
     MACRO zstring=ASCIIZ
     DECLARE FUNCTION o2_asm       LIB "oxygen.dll" ALIAS "o2_asm"       (zstring) AS STRING ' assemble
     DECLARE FUNCTION o2_link      LIB "oxygen.dll" ALIAS "o2_link"      (zstring) AS STRING ' o2 link **
     DECLARE FUNCTION o2_view      LIB "oxygen.dll" ALIAS "o2_view"      (zstring) AS STRING ' o2 coding
     DECLARE FUNCTION o2_asm_file  LIB "oxygen.dll" ALIAS "o2_asm_file"  (zstring) AS STRING ' assemble from file
     DECLARE FUNCTION o2_view_file LIB "oxygen.dll" ALIAS "o2_view_file" (zstring) AS STRING ' o2 coding from file
     DECLARE FUNCTION o2_error     LIB "oxygen.dll" ALIAS "o2_error"     ()        AS STRING ' error message if any
     DECLARE FUNCTION o2_len       LIB "oxygen.dll" ALIAS "o2_len"       ()        AS LONG   ' length of returned string
     ' ** o2_link is automatically called by o2_asm when the assembly is successful.

FUNCTION PBMAIN()

    DIM p   AS BYTE PTR
    DIM src AS STRING
    DIM cod AS STRING
    DIM lst AS STRING
    DIM erm AS STRING
    DIM r AS LONG
    DIM i AS LONG
    src="inc eax"+$CR+"ret"            ' minimal source code
    lst=LEFT$(o2_view((src)),o2_len()) ' view assembled code
    cod=LEFT$(o2_asm((src)),o2_len())  ' compile code string
    erm=LEFT$(o2_error(),o2_len())     ' check for any error
    IF erm="" THEN
        i=41
        p=STRPTR(cod)                  ' get pointer to code string
        ! mov eax,i                    ' pass a parameter
        ! call p ' execute code        ' call the code
        ! mov r,eax                    ' store the result
    END IF
    MSGBOX "Result:"+STR$(r)+$CR+$CR+lst

   FUNCTION=0

END FUNCTION


Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Theo Gottwald on April 25, 2008, 05:40:54 AM

Hi Charles

I have seen this in the supplied example. If you do a lot of work, why do you leave away the last mile to make it really comfortable :-)?

Just assume that people want to use it, but do not need to use it (mostly there are more possible ways to go).
The the "ease of use" is what counts.

Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Charles Pegge on April 25, 2008, 07:34:56 AM

Hi Theo,

You  must bear in mind that the above example contains the entire header for Oxygen, customised for PB. It's up to the client program, how the module is used.

Here is a minimalist example ( removing all the luxury items :) ). In this example, we leave the assembled code sitting in the buffer and call it directly. So the assembly process boils down to

p=asmo(src)

then calling:

CALL DWORD p

Code: [Select]

DECLARE FUNCTION asmo       LIB "oxygen.dll" ALIAS "o2_asm"       (ASCIIZ) AS LONG ' assemble

FUNCTION PBMAIN()

    DIM src AS ASCIIZ*256
    DIM cod AS STRING
    DIM p AS LONG
    DIM r AS LONG
    DIM i AS LONG
    src="inc eax"+$CR+"ret"            ' minimal source code
    p=asmo(src)
    i=41
    ! mov eax,i                    ' pass a parameter
    ! call p ' execute code        ' call the code
    ! mov r,eax                    ' store the result
    MSGBOX "Result:"+STR$(r)

END FUNCTION


Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Charles Pegge on April 25, 2008, 11:46:15 AM

Taking the buffer idea a stage further, I've added an O2_buf(#id) function which allows up to 512 pieces of assembled code to be retained by the Oxygen module. Then there is no need for the client to retrieve strings from a single buffer.

I've updated Oxygen.zip at the start of this thread, with the new function and a few other internal tweaks to facilitate module-sided buffering.

Code: [Select]

    'PB interface
     DECLARE FUNCTION asmo       LIB "oxygen.dll" ALIAS "o2_asm"       (ASCIIZ) AS LONG ' assemble
     DECLARE FUNCTION bufo       LIB "oxygen.dll" ALIAS "o2_buf"       (BYVAL LONG)AS LONG   ' buffer selector (returns pointer)

FUNCTION PBMAIN()

    LOCAL p,r,i AS LONG
    bufo 23                   ' set the buffer  number to 23
    asmo "inc eax"+$CR+"ret"  ' assemble minimal source code
    '....
    p=bufo(23)                '... recall pointer for buffer 23

    ' use the code

    i=41
    ! mov eax,i                    ' pass a parameter
    ! call p
    ! mov r,eax                    ' store the result

    MSGBOX "Result:"+STR$(r)

   FUNCTION=0

END FUNCTION


Title: Re: Asmosphere Dynamic assembler with SSE2 SSE3 SSE4 and 3DNOW
Post by: Charles Pegge on April 26, 2008, 01:13:54 PM

There is now a Linux version of Oxygen posted at the top of this thread. Using FB .185 I was unable to get the demo program to link to Oxygen at compile time so I switched to run-time linking which is probably a more robust and flexible way to do things in the long run.

Title: Re: Asmosphere Dynamic assembler - with editable instruction set
Post by: Charles Pegge on April 30, 2008, 05:03:06 PM

I've soft coded about 250 instructions, including all the SSE and 3dNow instructions. These have been moved from the kernel of the assembler into an editable file called o2asm.data. This will make maintenance for new instructions and alternative names very simple.

The assembler is now smaller and faster, using a hash algorithm and linked lists to lookup instructions.