Author Topic: FileSystem Enum and Roaming  (Read 7119 times)

0 Members and 1 Guest are viewing this topic.

Offline Theo Gottwald

  • Administrator
  • Hero Member
  • *****
  • Posts: 1058
  • User-Rate: +30/-4
  • Gender: Male
    • it-berater
FileSystem Enum and Roaming
« on: June 28, 2014, 12:00:41 PM »
Enumerating the Filesystem C: with Flags to include the System-Folders, i get the same folder twice,

First I get this:
C:\Users\All Users\Microsoft\Search\Data\Applications\Windows\MSStmp.log

Later i get this.
C:\ProgramData\Microsoft\Search\Data\Applications\Windows\MSStmp.log

Whats the trick to avoid that?
I only want to enumerate each file once ...

Offline Pierre Bellisle

  • JK-Mitglieder
  • Full Member
  • *
  • Posts: 149
  • User-Rate: +12/-7
Re: FileSystem Enum and Roaming
« Reply #1 on: June 28, 2014, 02:34:16 PM »
Hey Theo,

Check for the "reparse point or symbolic link" file attribute...

Pierre

[Updated: Minor change]
Code: [Select]
#COMPILE EXE '#Win 8.04#
#DIM ALL
#INCLUDE "Win32Api.inc"

%GETFILEEXINFOSTANDARD           = 0
%FILE_ATTRIBUTE_INTEGRITY_STREAM = &h00008000??? 'Windows Server 2012
%FILE_ATTRIBUTE_NO_SCRUB_DATA    = &h00020000??? 'Windows 8
%FILE_ATTRIBUTE_VIRTUAL          = &h00010000???
%FILE_ATTRIBUTE_VOLUME           = &h00000008???
'______________________________________________________________________________

FUNCTION AttrToString(sFile AS STRING) AS STRING
 LOCAL FileData AS WIN32_FILE_ATTRIBUTE_DATA
 LOCAL sAttr AS STRING
 LOCAL Attr  AS DWORD

 GetFileAttributesEx(BYVAL STRPTR(sFile), %GETFILEEXINFOSTANDARD, BYVAL VARPTR(FileData))
 Attr = FileData.dwFileAttributes

 IF ERR = 0 THEN
   IF Attr = 0                                       THEN sAttr = "normal-0, " '0 (0x0)
   IF (Attr AND %FILE_ATTRIBUTE_VOLUME)              THEN sAttr = "volume, " '8 (0x8)
   IF (Attr AND %FILE_ATTRIBUTE_DIRECTORY)           THEN sAttr = sAttr & "folder, " ELSE sAttr = sAttr & "file, " '16 (0x10)
   IF (Attr AND %FILE_ATTRIBUTE_HIDDEN)              THEN sAttr = sAttr & "hidden, " '2 (0x2)
   IF (Attr AND %FILE_ATTRIBUTE_READONLY)            THEN sAttr = sAttr & "read only, " '1 (0x1)
   IF (Attr AND %FILE_ATTRIBUTE_SYSTEM)              THEN sAttr = sAttr & "system, " '4 (0x4)
   IF (Attr AND %FILE_ATTRIBUTE_ARCHIVE)             THEN sAttr = sAttr & "archive, " '32 (0x20)
   IF (Attr AND %FILE_ATTRIBUTE_NORMAL)              THEN sAttr = sAttr & "normal-80, " '128 (0x80)

   IF (Attr AND %FILE_ATTRIBUTE_COMPRESSED)          THEN sAttr = sAttr & "compressed, "  '2048 (0x800)
   IF (Attr AND %FILE_ATTRIBUTE_DEVICE)              THEN sAttr = sAttr & "device, " '64 (0x40)
   IF (Attr AND %FILE_ATTRIBUTE_ENCRYPTED)           THEN sAttr = sAttr & "encrypted, " '16384 (0x4000)
   IF (Attr AND %FILE_ATTRIBUTE_INTEGRITY_STREAM)    THEN sAttr = sAttr & "stream, " '32768 (0x8000)
   IF (Attr AND %FILE_ATTRIBUTE_NOT_CONTENT_INDEXED) THEN sAttr = sAttr & "not to be indexed, " '8192 (0x2000)
   IF (Attr AND %FILE_ATTRIBUTE_NO_SCRUB_DATA)       THEN sAttr = sAttr & "no scrub data, " '131072 (0x20000)
   IF (Attr AND %FILE_ATTRIBUTE_OFFLINE)             THEN sAttr = sAttr & "offline, " '4096 (0x1000)
   IF (Attr AND %FILE_ATTRIBUTE_REPARSE_POINT)       THEN sAttr = sAttr & "reparse point or symbolic link, " '1024 (0x400)
   IF (Attr AND %FILE_ATTRIBUTE_SPARSE_FILE)         THEN sAttr = sAttr & "sparse file, " '512 (0x200)
   IF (Attr AND %FILE_ATTRIBUTE_TEMPORARY)           THEN sAttr = sAttr & "temporary file, " '256 (0x100
   IF (Attr AND %FILE_ATTRIBUTE_VIRTUAL)             THEN sAttr = sAttr & "virtual, " '65536 (0x10000)

   FUNCTION = LEFT$(sAttr, - 2)
 ELSE
   FUNCTION = "Path not found"
 END IF

END FUNCTION
'______________________________________________________________________________

FUNCTION PBMAIN() AS LONG
 LOCAL sBuffer  AS STRING
 LOCAL sFolder1 AS STRING
 LOCAL sFolder2 AS STRING

 sFolder1 = "C:\Users\All Users\"
 sFolder2 = "C:\ProgramData\"

 sBuffer  = "Directory ""reparse point / symbolic link"", those are the same folder..." & $CRLF & $CRLF & $CRLF & _
            sFolder1 & $CRLF & "Attributes:" & $CRLF & AttrToString(sFolder1)  & $CRLF & $CRLF & $CRLF & _
            sFolder2 & $CRLF & "Attributes:" & $CRLF & AttrToString(sFolder2)

 MessageBox(%HWND_DESKTOP, BYVAL STRPTR(sBuffer), "Reparse point or symbolic link", _
            %MB_ICONINFORMATION OR %MB_OK OR %MB_APPLMODAL)

END FUNCTION
'______________________________________________________________________________

'%FILE_ATTRIBUTE_ARCHIVE             '    32 (0x20)    A file or directory that is an archive file or directory. Applications typically use this attribute to mark files for backup or removal .
'%FILE_ATTRIBUTE_COMPRESSED          '  2048 (0x800)   A file or directory that is compressed. For a file, all of the data in the file is compressed. For a directory, compression is the default for newly created files and subdirectories.
'%FILE_ATTRIBUTE_DEVICE              '    64 (0x40)    This value is reserved for system use.
'%FILE_ATTRIBUTE_DIRECTORY           '    16 (0x10)    The handle that identifies a directory.
'%FILE_ATTRIBUTE_ENCRYPTED           ' 16384 (0x4000)  A file or directory that is encrypted. For a file, all data streams in the file are encrypted. For a directory, encryption is the default for newly created files and subdirectories.
'%FILE_ATTRIBUTE_HIDDEN              '     2 (0x2)     The file or directory is hidden. It is not included in an ordinary directory listing.
'%FILE_ATTRIBUTE_INTEGRITY_STREAM    ' 32768 (0x8000)  The directory or user data stream is configured with integrity (only supported on ReFS volumes). It is not included in an ordinary directory listing.
                                                      'The integrity setting persists with the file if it's renamed.
                                                      'If a file is copied the destination file will have integrity set if either the source file or destination directory have integrity set.
                                                      'Windows Server 2008 R2, Windows 7, Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP:  This flag is not supported until Windows Server 2012.
'%FILE_ATTRIBUTE_NORMAL              '   128 (0x80)    A file that does not have other attributes set. This attribute is valid only when used alone.
'%FILE_ATTRIBUTE_NOT_CONTENT_INDEXED '  8192 (0x2000)  The file or directory is not to be indexed by the content indexing service.
'%FILE_ATTRIBUTE_NO_SCRUB_DATA       '131072 (0x20000) The user data stream not to be read by the background data integrity scanner (AKA scrubber). When set on a directory it only provides inheritance.
                                                      'This flag is only supported on Storage Spaces and ReFS volumes. It is not included in an ordinary directory listing.
                                                      'Windows Server 2008 R2, Windows 7, Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP:  This flag is not supported until Windows 8 and Windows Server 2012.
'%FILE_ATTRIBUTE_OFFLINE             '  4096 (0x1000)  The data of a file is not available immediately. This attribute indicates that the file data is physically moved to offline storage. This attribute is used by Remote Storage,
                                                      'which is the hierarchical storage management software. Applications should not arbitrarily change this attribute.
'%FILE_ATTRIBUTE_READONLY            '     1 (0x1)     A file that is read-only. Applications can read the file, but cannot write to it or delete it. This attribute is not honored on directories.
                                                      'For more information, see You cannot view or change the Read-only or the System attributes of folders in Windows Server 2003, in Windows XP, in Windows Vista or in Windows 7.
'%FILE_ATTRIBUTE_REPARSE_POINT       '  1024 (0x400)   A file or directory that has an associated reparse point, or a file that is a symbolic link.
'%FILE_ATTRIBUTE_SPARSE_FILE         '   512 (0x200)   A file that is a sparse file.
'%FILE_ATTRIBUTE_SYSTEM              '     4 (0x4)     A file or directory that the operating system uses a part of, or uses exclusively.
'%FILE_ATTRIBUTE_TEMPORARY           '   256 (0x100)   A file that is being used for temporary storage. File systems avoid writing data back to mass storage if sufficient cache memory is available,
                                                      'because typically, an application deletes a temporary file after the handle is closed. In that scenario, the system can entirely avoid writing the data.
                                                      'Otherwise, the data is written after the handle is closed.
'%FILE_ATTRIBUTE_VIRTUAL             ' 65536 (0x10000) This value is reserved for system use.
'______________________________________________________________________________
'
« Last Edit: June 28, 2014, 03:59:16 PM by Pierre Bellisle »

Offline Theo Gottwald

  • Administrator
  • Hero Member
  • *****
  • Posts: 1058
  • User-Rate: +30/-4
  • Gender: Male
    • it-berater
Re: FileSystem Enum and Roaming
« Reply #2 on: June 29, 2014, 08:56:37 AM »
Thanks, Pierre. I'll try ...

Offline Pierre Bellisle

  • JK-Mitglieder
  • Full Member
  • *
  • Posts: 149
  • User-Rate: +12/-7
Re: FileSystem Enum and Roaming
« Reply #3 on: June 29, 2014, 12:52:30 PM »
To expand a little, in this particular case, you have to get the attribute
of all folders of the tree, C:\Users", "C:\Users\All Users",
"C:\Users\All Users\Microsoft", etc. to see
if one have the "reparse point or symbolic link" file attribute.
As soon you find one, your enumeration should skip what's under it
becose everything from that point could be seen twice in your files listing.

Since "C:\Users\All Users\" have this attribute the operating system
will get the corresponding folder and jump in it, meaning "C:\ProgramData\".
From that point, every files and folders seen under "C:\Users\All Users\"
will be in reality, under "C:\ProgramData\".

In the code above, all is needed is to check for the attribute.
For simplyicity, I did not put the code to find the
corresponding folder of a given a specific directory.

Note that there is also the "localized filename", a different thing,
it's when you see, let's say in french, "C:\Utilisateurs"
instead of "C:\Users" in Windows Explorer...
"C:\Usuarios" ... "C:\Benutzer" ... Well, you get the picture.

Hope this help. :-)

Pierre
« Last Edit: June 29, 2014, 06:00:23 PM by Pierre Bellisle »

Offline Pierre Bellisle

  • JK-Mitglieder
  • Full Member
  • *
  • Posts: 149
  • User-Rate: +12/-7
Re: FileSystem Enum and Roaming
« Reply #4 on: June 29, 2014, 01:01:44 PM »
The MSDN remarks section of CreateSymbolicLink and CreateHardLink may also be helpfull.

Offline Pierre Bellisle

  • JK-Mitglieder
  • Full Member
  • *
  • Posts: 149
  • User-Rate: +12/-7
Re: FileSystem Enum and Roaming
« Reply #5 on: June 29, 2014, 10:45:44 PM »
For curious peoples...

Resolve "Reparse point and Symbolic link",
and also "Localized fileName"

Click on image to see code...

Pierre



« Last Edit: June 30, 2014, 02:43:30 AM by Pierre Bellisle »

Offline Theo Gottwald

  • Administrator
  • Hero Member
  • *****
  • Posts: 1058
  • User-Rate: +30/-4
  • Gender: Male
    • it-berater
Re: FileSystem Enum and Roaming
« Reply #6 on: July 01, 2014, 12:36:53 AM »
Pierre, thanks for that extensive answer.
It works fine, and it will also help other users with the same problem.

Offline Pierre Bellisle

  • JK-Mitglieder
  • Full Member
  • *
  • Posts: 149
  • User-Rate: +12/-7
Re: FileSystem Enum and Roaming
« Reply #7 on: July 01, 2014, 12:45:35 AM »
Great :-)

Pierre