Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 178

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 183

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 184

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 220

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 223

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 235

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 250

Deprecated: Function create_function() is deprecated in /homepages/21/d38531796/htdocs/jose/smfforum/Sources/Load.php on line 268
FreeBASIC CWstr

Author Topic: FreeBASIC CWstr  (Read 11452 times)

0 Members and 1 Guest are viewing this topic.

Offline Jeff Marshall

  • Newbie
  • *
  • Posts: 19
Re: FreeBASIC CWstr
« Reply #180 on: January 20, 2019, 05:26:07 PM »
Yes, it looks like it only has to do with line endings.

One part of the problem is how fbc project is configured compared to how your system is configured.  And maybe, you are using a .gitattributes file locally to try and manage this CRLF/LF problem.  Not sure.  Regardless, it's the line endings that are causing the grief.

The other part, is that last summer, I applied a patch from SARG and still got mixed line endings stored to the repo.  Some of the changed files you are seeing are all the files that got merged from pull request 96.  https://github.com/freebasic/fbc/pull/96 (like emit.bi edbg_stab.bas).  Where we are just now seeing the LF -> CRLF conversion.

So,

We do want every file we get from freebasic/fbc to have CRLF line endings on windows, and LF on linux.

From fbc wiki https://www.freebasic.net/wiki/wikka.php?wakka=DevGettingTheSourceCode
: The recommended setting for core.autocrlf is true, so that the FB source code in the working tree will have CRLF line endings.

Locally, for a single repo, looks like setting autocrlf can be stored in ./git/config file.  (git config --local).  Need to make sure that works like I think it should.

From this site: https://ywjheart.wordpress.com/2017/03/22/autocrlf/
: use core.autocrlf = true if you plan to use this project under Unix as well

Also, if the setting autocrlf has been changed, or in .gitattributes, then the changes may not get seen right away
Info in https://help.github.com/articles/dealing-with-line-endings/

heh, I'm trying.  Also, the development tree is shared by many people, so I keep most scripts, batch files, IDE files, tools, etc, outside of the fbc directory.

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #181 on: January 20, 2019, 07:35:46 PM »
Well, you can set autocrlf systemwide, global and local. All of these settings end up in a config file in the corresponding location. As i understand it setting autocrlf to true doesn´t affect .bas/.bi files, it´s for .txt files (at least for me it doesn´t convert -bas/.bi files at all - i tried it). Setting it to false removes all .txt files from the list of files GIT reports to have been modified. So i don´t have to deal with them anymore.

My IDE cannot work with LF line endings (which btw isn´t Windows standard), so i must convert them to CRLF. I tried .gitattribute, but you told me not to use it. Now i wrote a little utility doing that for me (all files in src\compile and all new files i added). Copying these files (with CRLF line endings) to Linux (Ubuntu64 on a virtual machine), didn´t cause any problems there, neither did the test files.

Which files must have LF line endings ? Most files in src\compiler already have CRLF line endings anyway, it´s only a few, which i must convert. If i had a list of them, i could write an (expandable) utility for converting them from CRLF to LF (and vice versa) as needed. A simple .txt file for listing directories or single files could supply input and the utility would convert these files for Windows or Linux. What do you think ?


JK

Offline Jeff Marshall

  • Newbie
  • *
  • Posts: 19
Re: FreeBASIC CWstr
« Reply #182 on: January 20, 2019, 09:18:42 PM »
core.autocrlf affects all TEXT files.  Which could be ANY filename.ext, that does not have binary content.  With correct autocrlf setting, and no extra .gitignore or .gitattributes, I don't think you should have to write scripts to deal with line endings.

Easiest way to see the difference is do a new clone & checkout, forcing the autocrlf setting for comparison.

Clone & checkout into fb.0 with whatever line ending was stored in the repo (should be all LF):
git -c core.autocrlf=false  clone https://github.com/freebasic/fbc fb.0

Clone & checkout into fb.1, converting all TEXT files to CRLF line endings:
git -c core.autocrlf=true clone https://github.com/freebasic/fbc fb.1

autocrlf setting is also important when making commits back to fbc repo.
When committing to from windows, autocrlf=true (CRLF's get converted back to LF's)
When committing to from linux, autocrlf=false (line endings get stored as-is which should already have been LF)

The only files that must be LF are
./manual/cache/*.wakka (raw data files from wiki)
./tests/file/big-list-eol-lf.txt (a specific test that needs LF on all targets)

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #183 on: January 20, 2019, 09:58:20 PM »
Quote
Easiest way to see the difference is do a new clone & checkout, forcing the autocrlf setting for comparison.
that´s exactly what i did! Regardless of the autocrlf setting, i always have some files with LF line endings after cloning in src\compiler - i spend days trying to get GIT do what i want (CRLF as line endings right after cloning), it just didn´t work. Finally i gave up on it and it took me ten minutes to write own code fixing the issue.

Quote
The only files that must be LF are
./manual/cache/*.wakka (raw data files from wiki)
./tests/file/big-list-eol-lf.txt (a specific test that needs LF on all targets)
I didn´t touch any of these files, nor were any of them part of a commit, so there should be no problem.

What about just testing and evaluating my changes and additions, while i on my part work on a solution to preserve line endings just as they are in the master branch? Or why not have line endings for the source code consistent - either all LF or all CRLF, instead of having some with LF and having others with CRLF?

Or add a file e.g "rules.txt" specifying rules like naming conventions, required line endings, exceptions from the rule and the like for the root folders of the repo. This would make things clear once and for good.


JK

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #184 on: January 20, 2019, 11:11:50 PM »
Re-reading the info in https://help.github.com/articles/dealing-with-line-endings/, what about a .gitattributes file in the master branch? Properly configured this would solve the problems we have for good too, because .gitattributes did work for me - but you advised me not to use it. I understand now that my implementation was improper for general use, even if it worked for me.

But it should be possible to setup a generic .gitattributes file. Because of the fact that it overrides other GIT settings, it would make clones independent of individual user settings in this repect and ensure proper line endings everywhere.

Please, think about it.


JK

 
« Last Edit: January 20, 2019, 11:16:17 PM by Juergen Kuehlwein »

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #185 on: January 21, 2019, 10:44:20 PM »
Jeff,

i ran further tests showing that .gitattributes could really be the solution. As already mentioned above setting autocrlf to true doesn´t convert all .bas/.bi files properly to crlf - emit.bi and others keep failing!

Adding a .gitattributes at root level like this

Code: [Select]
# Auto detect text files and perform LF normalization
* text=auto
# BASIC source files (need cr/lf line endings)
*.bas text
*.bi text
*.inc text
*.h text
*.bat text
*.txt text
*.jkp text
*.rc text

*.gif binary
*.jpg binary
*.png binary

./manual/cache/*.wakka eol=lf
./tests/file/big-list-eol-lf.txt eol=lf

does the trick. Astonishingly enough all other problems i had with .txt and .sh files seem to be gone as well.

You may have to adapt and refine what i posted here, but in my opinion this the way to go in order to avoid the mess we currently have. It does what i already proposed above, it allows for setting rules for line endings per file type, it s independent of user settings because it overrides them. That is, it forces consistent line endings for the corresponding OS. On the contrary to autocrlf it works - and you can control it.


JK

Offline Jeff Marshall

  • Newbie
  • *
  • Posts: 19
Re: FreeBASIC CWstr
« Reply #186 on: January 24, 2019, 11:53:46 PM »
Hey JK, yes, I've seen all your updates.  It's only been a few days, dude!  Please allow me some time to review and respond.  I really want to get the 1.06 release out first.  Then we can be more liberal about what gets merged in to fbc/master.  Just after a release is a good time to add new features.

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #187 on: January 25, 2019, 03:17:16 PM »
Quote
I really want to get the 1.06 release out first

Yes, do that! (and then let´s go on ... :-))


JK

Offline Juergen Kuehlwein

  • Full Member
  • ***
  • Posts: 106
Re: FreeBASIC CWstr
« Reply #188 on: April 07, 2019, 11:41:32 AM »
Hi José,


after a little break here is an update of the current situation:

After the 1.06 FreeBASIC release Jeff basically agreed to merge in my pull request. There might be changes to some additions (extra string and array handling functions) but he definitely supports the integration of USTRING into the compiler´s code.

That is, we can finally get rid of the need to prepend "**" for variables and expressions at all. All of these changes work with my implemetation (CWSTR stripped down to the bare minimum) as well as with your original CWSTR. Existing code doesn´t have to be changed, you can still use "**", but you don´t need to anymore.


My code differs in two minor points from yours (CWSTR):
- i had to move "m_pBuffer" in first place of the member variables, this makes an improved "STRPTR" possible

Code: [Select]
TYPE DWSTR
  m_pBuffer AS UBYTE PTR                              'pointer to the buffer (in first place -> make strptr possible)

  Private:
    m_Capacity AS Ulong                               'the total size of the buffer
    m_GrowSize AS LONG = 260 * SizeOf(WSTRING)        'how much to grow the buffer by when required

  Public:
    m_BufferLen AS ulong                              'length in bytes of the current string in the buffer

STRPTR now returns a WSTRING PTR for Wstrings and Ustrings, which makes much more sense than a previously returned ZSTRING PTR. This change doesn´t break existing code, because you couldn´t use a ZSTRING PTR for processing a Wstring, you had to cast it to a WSTRING PTR or ANY PTR anyway.


- i had to remove the "CONST" qualifier in one place, this was necessary for some string handling functions to compile

Code: [Select]
    DECLARE OPERATOR CAST () BYREF AS WSTRING
    DECLARE OPERATOR CAST () AS ANY PTR


With these two small changes (which shouldn´t break anything) your CWSTR will be fully compatible to my compiler changes. Basically my USTRING implementation is written in a way, that (if pesent) WINFBX is the preferred way for adding dynamic wide strings. My code is only a fallback, if WINFBX is not used or cannot be used (e.g. LINUX).

Do you see any problems applying these changes to future releases of WINFBX?


JK

Offline José Roca

  • Administrator
  • Hero Member
  • *****
  • Posts: 2514
    • José Roca Software
Re: FreeBASIC CWstr
« Reply #189 on: April 08, 2019, 03:08:05 AM »
No problem at all.