Discussion:
Sandsifter software finds hidden instructions inside processors.
(too old to reply)
s***@hotmail.com
2017-10-21 15:10:09 UTC
Permalink
Hello,

I just watched this video, it's pretty interesting and somewhat amazing, this guy find a way to find hidden instructions inside processors:



The software which finds these hidden instructions is available too:

https://github.com/xoreaxeaxeax/sandsifter

Apperently it's open source python and a little bit of C.

It will probably require some kind of python interpreter/executor and probably admin rights to run.

I am not yet sure how to run this software it will require installing some additional capstone disassembler.

I am curious what would be found on my AMD X2 3800+ from almost 12 years ago... if it will run at all... I think it will run.

I would be curious to also here results of other people ! ;)

So if you curious as to what secret instructions exist inside your computer give this a run.

Later today or in the coming days when I have some time for this I will return to
this subject.

For now I may have other things to do or maybe not :P :)

Bye,
Skybuck =D
Rod Pemberton
2017-10-22 06:47:31 UTC
Permalink
On Sat, 21 Oct 2017 08:10:09 -0700 (PDT)
***@hotmail.com wrote:


On alt.lang.asm,
Post by s***@hotmail.com
Hello,
I just watched this video, it's pretty interesting and somewhat
amazing, this guy find a way to find hidden instructions inside
http://youtu.be/KrksBdWcZgQ
https://github.com/xoreaxeaxeax/sandsifter
Yes, someone posted about it to Hackaday a while back:

https://hackaday.com/?s=sandsifter

Some of his other projects have been mentioned before on comp.arch.
Post by s***@hotmail.com
Apperently it's open source python and a little bit of C.
It will probably require some kind of python interpreter/executor and
probably admin rights to run.
I am not yet sure how to run this software it will require installing
some additional capstone disassembler.
I am curious what would be found on my AMD X2 3800+ from almost 12
years ago... if it will run at all... I think it will run.
I would be curious to also here results of other people ! ;)
So if you curious as to what secret instructions exist inside your
computer give this a run.
Later today or in the coming days when I have some time for this I
will return to this subject.
For now I may have other things to do or maybe not :P :)
Bye,
Skybuck =D
Personally, I'm not interested in instructions which may only work on
processors of a certain generation or manufacturer, and which can't be
disabled. AISI, these can really only be useful for hacking, e.g.,
privilege escalation, or perhaps used to detect software emulation, or
perhaps used to prevent dis-assembly, i.e., obfuscation, or maybe used
in a graphics demo, e.g., for speed. All of these situations are low
use, specialty, or evil. Although, I'm sure some demo coders would
still be interested in knowing these for the original 8086.

What I'm interested in are instructions which work consistently,
reliably on all x86/AA-64 processors for the respective processor mode.
AISI, these are useful for ensuring that compilers and assemblers
generate code which runs consistently on every processor when executing
code for the specific processor mode. Most of these should be the
instructions documented in the Intel and AMD manuals. But, as you're
aware, some processors had bugs in the known, standard instructions.
This also implies that I'm interested in instructions which were
implemented and then obsoleted.


Rod Pemberton
--
Why are Hillary's living donors pedophiles and rapists? Bill Clinton,
Harvey Weinstein, Anthony Wiener, Jeffrey Epstein, ...
s***@hotmail.com
2017-10-23 02:11:23 UTC
Permalink
Post by Rod Pemberton
On Sat, 21 Oct 2017 08:10:09 -0700 (PDT)
On alt.lang.asm,
Post by s***@hotmail.com
Hello,
I just watched this video, it's pretty interesting and somewhat
amazing, this guy find a way to find hidden instructions inside
http://youtu.be/KrksBdWcZgQ
https://github.com/xoreaxeaxeax/sandsifter
https://hackaday.com/?s=sandsifter
Some of his other projects have been mentioned before on comp.arch.
Post by s***@hotmail.com
Apperently it's open source python and a little bit of C.
It will probably require some kind of python interpreter/executor and
probably admin rights to run.
I am not yet sure how to run this software it will require installing
some additional capstone disassembler.
I am curious what would be found on my AMD X2 3800+ from almost 12
years ago... if it will run at all... I think it will run.
I would be curious to also here results of other people ! ;)
So if you curious as to what secret instructions exist inside your
computer give this a run.
Later today or in the coming days when I have some time for this I
will return to this subject.
For now I may have other things to do or maybe not :P :)
Bye,
Skybuck =D
Hi,

Thank you for your somewhat interesting reply !
Post by Rod Pemberton
Personally, I'm not interested in instructions which may only work on
processors of a certain generation or manufacturer, and which can't be
disabled.
I do find it interesting to see what kind of secret instructions have been added or removed to certain generations.

However you will be "happy" to hear that this SandSifter software has found *the same* hidden instructions which are present on Intel, AMD and even Transmeta processors ?!

This is what has me worried. Why would there be such a thing as "shared hidden instructions" across different hardware ?

It's either some kind of debugging/production manufacturing thing or it might be something more sinister.
Post by Rod Pemberton
AISI, these can really only be useful for hacking, e.g.,
privilege escalation, or perhaps used to detect software emulation, or
Interesting idea to use these for "software emulation" detection ! :)
Post by Rod Pemberton
perhaps used to prevent dis-assembly, i.e., obfuscation, or maybe used
in a graphics demo, e.g., for speed. All of these situations are low
use, specialty, or evil.
Indeed, more uses might be found and for now it is unknown what these hidden instructions do or could/might be used for.
Post by Rod Pemberton
Although, I'm sure some demo coders would
still be interested in knowing these for the original 8086.
Precisely my thoughts as well. I would be very interested to know if 80486 for example also has (many?) hidden instructions and also how to detect them.

The "no execute" functionality is not present, though perhaps the Undefined Exception might be used to detect if a "random/trail/hidden" instruction is present or not.

One possible approach to testing virtual machine could be to download Linux Mint 18.2 32bit edition and then try and boot it from ISO in VMWare... and then run this software to see how VMWare handles it ! Pretty interesting idea... I might try this next ! ;) :)

Then again... perhaps it's a stupid idea because maybe 32 bit mode does not have "no execute" bit in page table. However somehow this software might still be able to detect instructions, so it might be interesting to try it anyway ! :)
Post by Rod Pemberton
What I'm interested in are instructions which work consistently,
reliably on all x86/AA-64 processors for the respective processor mode.
I am not sure if the tool outputs which modes the instructions work on, it might, you may have to inspect this software further to see if it differentiates between processor modes.

To be honest I am quite amazed that this software can detect ring 0 and ring -1 and ring -2 instructions by using this smart "no execute" trick.

I think this may have surprised Intel/AMD engineers as well.

It would seem a bit more logical to keep special instructions completely hidden from less privileged rings. So one could conclude that the priority/precedence and "error messaging" might be at fault... wrongly implemented... it is leaking too much information/side channel information, for once I am happy about this for now :)

For "higher" privileged instructions the processor could have remained silent as to not leak any error information to lower level/privileges. On the other hand... the lower level would then not know that it does not have enough privileges so I do see a bit of a "debugging problem" there ;).

Apperently one could consider this a "catch-22" :) Damned if you do, damned if you don't :)

Perhaps a matter of taste. Suppose I would call the "White House" and ask if the "President was there ?" What would they answer ?

The answer might be: "None of your damn bussiness ?!" (for security reasons ?!?)

Strangely enough the Queen of England used to "Raise the Flag" when she was "In" :) Isn't that nuts ?! :) But Apperently she completely relies on her security to keep her Safe.

Apperently The Queen of England is a big believer in "Security and not Obscurity" ! :)

I am sure she will be applauded for that by some ! LOL. I bet England had many bloody wars and deaths and assessinations, so I am pretty sure the Queen knows what she is doing ! LOL.

Perhaps we can all learn a lesson from The Queen of England... Though nowadays after the whole "Lady Die" affair apperently raising the flag has lost it's meaning ? Hmmm Maybe she ran scared ? Who knows ;) :)

The issue was: The flag had to go "half-way" to morn for Lady Die's death... and it was against protocol.
Post by Rod Pemberton
AISI, these are useful for ensuring that compilers and assemblers
generate code which runs consistently on every processor when executing
code for the specific processor mode.
What is to say that these hidden instructions won't work reliably ? If you don't try you won't know... It's not like those wanting to keep this secret are going to tell you !?

Though now that the cat is out of the bag.. maybe some information will be forth coming...

This is an interesting point though.

If no further information is forth coming from these manufacturers this could then be interpreted as a "bad sign"... "a very bad sign".

Why would they not be "forth coming" with "additional information" if they have nothing to hide ?

Why keep a growing number of people in "uncertainty" ?

See where this is going if the situation of lack of information persists ?
Post by Rod Pemberton
Most of these should be the
instructions documented in the Intel and AMD manuals. But, as you're
aware, some processors had bugs in the known, standard instructions.
This also implies that I'm interested in instructions which were
implemented and then obsoleted.
Well SandSifter has proven one thing so far, the documentation is far from telling us everything about the instruction set and these processors.

This tool has found at least 800.000+ undocumented instructions ! That's quite a lot ! (Many of which are variations, but it's still quite a lot !)

Bye,
Skybuck.
Rod Pemberton
2017-10-23 04:13:44 UTC
Permalink
On Sun, 22 Oct 2017 19:11:23 -0700 (PDT)
Post by s***@hotmail.com
Post by Rod Pemberton
On Sat, 21 Oct 2017 08:10:09 -0700 (PDT)
AISI, these are useful for ensuring that compilers and assemblers
generate code which runs consistently on every processor when
executing code for the specific processor mode.
What is to say that these hidden instructions won't work reliably ?
Nothing. It was just an assumption by me that the designs would be
different.

However, your earlier statement may indicate that the hardware designs
are too similar for some unknown reasons (perhaps copying, cloning,
etc):

S> However you will be "happy" to hear that this SandSifter software has
S> found *the same* hidden instructions which are present on Intel, AMD
S> and even Transmeta processors ?!

If there is an instruction which can reliably detect real hardware
instead of software emulation, I'd be interested in reading about it.
Post by s***@hotmail.com
This tool has found at least 800.000+ undocumented instructions !
Most programmers seem to think of the instruction set as something
which is precisely defined. I.e., there shouldn't be any secret or
garbage instructions.

However, electrical engineers probably implemented the logic for
decoding the instructions as a matrix or multiple matrices. I.e.,
certain bits in instructions activate certain processor logic or
sequencing. So, some instructions have bits which produce a useful
result from the chained sequences and logic, whereas others just combine
some logic or sequencing and result in not doing much of anything
useful. Of course, these "useless" instructions still do something.
The question is, "What do they do?" These are likely the "hidden"
instructions, i.e., logic sequences mostly deemed to be "garbage" or
redundant by the engineers, etc.

Of course, you could do an analysis of the 800+ instructions. Maybe,
that will reveal more about the logic or micro-code sequences they use
to construct instructions. This could be used to generate more
accurate software emulation. That's probably valuable to someone.


Rod Pemberton
--
Why are Hillary's living donors pedophiles and rapists? Bill Clinton,
Harvey Weinstein, Anthony Wiener, Jeffrey Epstein, ...
James Harris
2017-10-23 11:35:51 UTC
Permalink
Post by Rod Pemberton
On Sun, 22 Oct 2017 19:11:23 -0700 (PDT)
Post by s***@hotmail.com
Post by Rod Pemberton
On Sat, 21 Oct 2017 08:10:09 -0700 (PDT)
AISI, these are useful for ensuring that compilers and assemblers
generate code which runs consistently on every processor when
executing code for the specific processor mode.
What is to say that these hidden instructions won't work reliably ?
Nothing. It was just an assumption by me that the designs would be
different.
There is always the "contract" which is the set of instruction
specifications in the manuals. Any undocumented instructions may work on
lots of processors but are not guaranteed to work on all of them. That's
only true of the documented ones (errata permitting).

Similarly, one could test a certain instruction on a load of processors
and find out what effects it has on flags. But there's no guarantee that
the same effects will apply to other CPUs. Hence, the effects are
documented as "undefined".
--
James Harris
wolfgang kern
2017-10-23 08:44:19 UTC
Permalink
***@hotmail.com wrote:

[...]

|Well SandSifter has proven one thing so far, the documentation is far from
|telling us everything about the instruction set and these processors.
|
|This tool has found at least 800.000+ undocumented instructions ! That's
|quite a lot ! (Many of which are variations, but it's still quite a lot !)

just show me two opcodes (in hexadecimal notation and with 16/32/64 hints)
which you think are hidden and do something w/o raising an exception.

So I could either confirm it if I cant find them in my list, or I may show you
where they are mentioned to be alias or marked as unpredictable behave.

800000 ? this could mean you missed whole groups in your docs.
__
wolfgang
s***@hotmail.com
2017-10-24 16:00:53 UTC
Permalink
Post by wolfgang kern
[...]
|Well SandSifter has proven one thing so far, the documentation is far from
|telling us everything about the instruction set and these processors.
|
|This tool has found at least 800.000+ undocumented instructions ! That's
|quite a lot ! (Many of which are variations, but it's still quite a lot !)
just show me two opcodes (in hexadecimal notation and with 16/32/64 hints)
which you think are hidden and do something w/o raising an exception.
So I could either confirm it if I cant find them in my list, or I may show you
where they are mentioned to be alias or marked as unpredictable behave.
800000 ? this could mean you missed whole groups in your docs.
__
wolfgang
Hello wolfgang,

The SandSifter program has collected all suspicious instructions, there byte sequences are in the following file (this could include prefix bytes, only 1 though):

http://www.skybuck.org/SandSifter/unzipped/data/log

The file is 180 megabytes. Is this too large for you to download ?

What do you mean with 16/32/64 hints ?

Anyway here are some byte sequences from the file:

0f0d00
0f0d0400
0f0d040500000000
2e0f0d1485b9000000
420f0d341e
640f0d5e21
660f38

Bye,
Skybuck.
wolfgang kern
2017-10-24 19:59:32 UTC
Permalink
Post by s***@hotmail.com
Post by wolfgang kern
[...]
|Well SandSifter has proven one thing so far, the documentation is far from
|telling us everything about the instruction set and these processors.
|
|This tool has found at least 800.000+ undocumented instructions ! That's
|quite a lot ! (Many of which are variations, but it's still quite a lot !)
just show me two opcodes (in hexadecimal notation and with 16/32/64 hints)
which you think are hidden and do something w/o raising an exception.
So I could either confirm it if I cant find them in my list, or I may show you
where they are mentioned to be alias or marked as unpredictable behave.
800000 ? this could mean you missed whole groups in your docs.
__
wolfgang
Hello wolfgang,
The SandSifter program has collected all suspicious instructions, there byte
sequences are in the following file (this could include prefix bytes, only 1
http://www.skybuck.org/SandSifter/unzipped/data/log
The file is 180 megabytes. Is this too large for you to download ?
yes because I heavy suspect much sense in it ...
Post by s***@hotmail.com
What do you mean with 16/32/64 hints ?
thanks, I show what my disassembler tells about them:
... seems my daubts were right :)
Post by s***@hotmail.com
0f0d00
use16:
0f 0d 00 PREFETCH byte[bx+si]
use32:
0f 0d 00 PREFETCH byte[eax]
use64:
0f 0d 00 PREFETCH byte[rax]
Post by s***@hotmail.com
0f0d0400
use16:
0f 0d 04 PREFETCH byte[si] ;04 isn't an SIB in 16bit
use32:
0f 0d 04 00 PREFETCH byte[eax+eax]
use64:
0f 0d 04 00 PREFETCH byte[rax+rax]
Post by s***@hotmail.com
0f0d040500000000
use16: as above
use32:
0f 0d 04 05 00... PREFETCH byte[eax+00000000]
use64:
0f 0d 04 05 00... PREFETCH byte[rax+00000000]
Post by s***@hotmail.com
2e0f0d1485b9000000
use16:
2e CS:
0f0d14 is a redundant alias for 0f0d04
use32:
2e0f0d1485b900.. PREFETCH byte CS:[000000b9+eax*4]
Post by s***@hotmail.com
420f0d341e
use16/32: 42 is inc dx/edx
use64:
420f0d341e PREFETCH byte[rsi+r11]

0f0d341e is an alias for 0f0d041e, redundant bits were ignored.
Post by s***@hotmail.com
640f0d5e21
same here, redundant bit4 is ignored
use16/32/64:
640f0d4e21 PREFETCH byte FS:[(r,e)si+21]
Post by s***@hotmail.com
660f38
0f38 is an opcode group selector, the 66 will be ignored.

you can check for details on sandpile.org
__
wolfgang
s***@hotmail.com
2017-10-25 01:14:50 UTC
Permalink
Post by wolfgang kern
Post by s***@hotmail.com
Post by wolfgang kern
[...]
|Well SandSifter has proven one thing so far, the documentation is far from
|telling us everything about the instruction set and these processors.
|
|This tool has found at least 800.000+ undocumented instructions ! That's
|quite a lot ! (Many of which are variations, but it's still quite a lot !)
just show me two opcodes (in hexadecimal notation and with 16/32/64 hints)
which you think are hidden and do something w/o raising an exception.
So I could either confirm it if I cant find them in my list, or I may show you
where they are mentioned to be alias or marked as unpredictable behave.
800000 ? this could mean you missed whole groups in your docs.
__
wolfgang
Hello wolfgang,
The SandSifter program has collected all suspicious instructions, there byte
sequences are in the following file (this could include prefix bytes, only 1
http://www.skybuck.org/SandSifter/unzipped/data/log
The file is 180 megabytes. Is this too large for you to download ?
yes because I heavy suspect much sense in it ...
Post by s***@hotmail.com
What do you mean with 16/32/64 hints ?
... seems my daubts were right :)
Post by s***@hotmail.com
0f0d00
0f 0d 00 PREFETCH byte[bx+si]
0f 0d 00 PREFETCH byte[eax]
0f 0d 00 PREFETCH byte[rax]
Post by s***@hotmail.com
0f0d0400
0f 0d 04 PREFETCH byte[si] ;04 isn't an SIB in 16bit
0f 0d 04 00 PREFETCH byte[eax+eax]
0f 0d 04 00 PREFETCH byte[rax+rax]
Post by s***@hotmail.com
0f0d040500000000
use16: as above
0f 0d 04 05 00... PREFETCH byte[eax+00000000]
0f 0d 04 05 00... PREFETCH byte[rax+00000000]
Post by s***@hotmail.com
2e0f0d1485b9000000
0f0d14 is a redundant alias for 0f0d04
2e0f0d1485b900.. PREFETCH byte CS:[000000b9+eax*4]
Post by s***@hotmail.com
420f0d341e
use16/32: 42 is inc dx/edx
420f0d341e PREFETCH byte[rsi+r11]
0f0d341e is an alias for 0f0d041e, redundant bits were ignored.
Post by s***@hotmail.com
640f0d5e21
same here, redundant bit4 is ignored
640f0d4e21 PREFETCH byte FS:[(r,e)si+21]
Post by s***@hotmail.com
660f38
0f38 is an opcode group selector, the 66 will be ignored.
you can check for details on sandpile.org
__
wolfgang
Hi wolfgang !

Good find ! It does seem to be the prefetch instruction in all of it's form.

Sadly it's a bug in the capstone disassembler and very sadly it's still not fixed:

https://github.com/aquynh/capstone/issues/1012

I am not yet sure which version of capstone was installed during my experiment/run... fortunately I captured all output... so I should be able to figure out which version was used. Even if it's not yet fixed I might be able to use to fix as mention in the above link... hopefully this will not introduce any new problems.

Failing that, another approach could be to filter the 180 megabyte log file and illiminate/remove all prefetch instructions to see if any other kind of instructions remain.

Thanks for your help it has shed some more light on this ! =D

Bye,
Skybuck.
s***@hotmail.com
2017-10-25 02:34:50 UTC
Permalink
Hi wolfgang,

I found this online disassembler to help me make more sense of it.

https://www.onlinedisassembler.com/odaweb/6yBSvBOX/0

It seems to choke on this one:

64 0F 0F 0C CC 4D

It was reported by SandSifter summary as:

0F 0F 0C CC 4D

I found it on one of the "problem" screenshots so that's where I got it from, from the summarization tool... so even though it doesn't run on windows and crashes on linux mint 18.2 it was still of some use ? (I might try and get it too work on windows... but now I kinda understand why it might be crashing... because of all these "prefetch" instructions, though I am not sure if this one is a prefetch instruction too).

Loading Image...

Could you see if you can decode this one successfully ? :)

Bye,
Skybuck.
wolfgang kern
2017-10-25 09:12:30 UTC
Permalink
<***@hotmail.com>wrote:
|Hi wolfgang,
|
|I found this online disassembler to help me make more sense of it.
|
|https://www.onlinedisassembler.com/odaweb/6yBSvBOX/0

I wrote my own Dis-Ass, based on AMD-manuals.

|It seems to choke on this one:
|
|64 0F 0F 0C CC 4D

64 segover FS:
0F 0F tells it's a 3Dnow instruction.
0C 16bit: r/m
32/64: an SIB
CC 16bit: seen as imm8 which is part of the opcode
32/64: SIB modified r/m
4D 32/64: the imm8 opcode part

neither CC nor 4D opcode addons are in my 3Dnow list.

I just show 32bit yet:
640f0f 0c cc 0d PI2FD mm1,qword FS:[esp+ecx*8]

so the 4D might be an alias for 0D on some CPUs, it crashes here and my
disassembler correct reports: "EXC06, undefined 3Dnow" on it.

|It was reported by SandSifter summary as:
|
|0F 0F 0C CC 4D

|I found it on one of the "problem" screenshots so that's where I got it
|from, from the summarization tool... so even though it doesn't run on
|windows and crashes on linux mint 18.2 it was still of some use ? (I might
|try and get it too work on windows... but now I kinda understand why it
|might be crashing... because of all these "prefetch" instructions, though I
|am not sure if this one is a prefetch instruction too).

|Loading Image...

|Could you see if you can decode this one successfully ? :)

Sorry, I work with text-only here. If you want me to decode a few then just
post them here.

your count of 800+ undocumented instructions seem to miss many SSE3/4/5,
AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A 'prefixes'. I see only a
handful (~20) 'undocumented' (alias or useless anyway) opcodes.
__
wolfgang
Robert Wessel
2017-10-25 16:14:09 UTC
Permalink
Post by wolfgang kern
|Hi wolfgang,
|
|I found this online disassembler to help me make more sense of it.
|
|https://www.onlinedisassembler.com/odaweb/6yBSvBOX/0
I wrote my own Dis-Ass, based on AMD-manuals.
|
|64 0F 0F 0C CC 4D
0F 0F tells it's a 3Dnow instruction.
0C 16bit: r/m
32/64: an SIB
CC 16bit: seen as imm8 which is part of the opcode
32/64: SIB modified r/m
4D 32/64: the imm8 opcode part
neither CC nor 4D opcode addons are in my 3Dnow list.
640f0f 0c cc 0d PI2FD mm1,qword FS:[esp+ecx*8]
so the 4D might be an alias for 0D on some CPUs, it crashes here and my
disassembler correct reports: "EXC06, undefined 3Dnow" on it.
|
|0F 0F 0C CC 4D
|I found it on one of the "problem" screenshots so that's where I got it
|from, from the summarization tool... so even though it doesn't run on
|windows and crashes on linux mint 18.2 it was still of some use ? (I might
|try and get it too work on windows... but now I kinda understand why it
|might be crashing... because of all these "prefetch" instructions, though I
|am not sure if this one is a prefetch instruction too).
|http://www.skybuck.org/SandSifter/unzipped/problem/Screenshot%20from%|202017-10-22%2016-53-58.png
|Could you see if you can decode this one successfully ? :)
Sorry, I work with text-only here. If you want me to decode a few then just
post them here.
your count of 800+ undocumented instructions seem to miss many SSE3/4/5,
AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A 'prefixes'. I see only a
handful (~20) 'undocumented' (alias or useless anyway) opcodes.
Didn't the OP claim to be testing an AMD X2 3800+? I think that had
SSE3, but not SSE4/5/AVX etc.
wolfgang kern
2017-10-25 16:54:44 UTC
Permalink
Robert Wessel mentioned:
[...]
Post by Robert Wessel
Post by wolfgang kern
your count of 800+ undocumented instructions seem to miss many SSE3/4/5,
AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A 'prefixes'. I see
only a handful (~20) 'undocumented' (alias or useless anyway) opcodes.
Didn't the OP claim to be testing an AMD X2 3800+? I think that had
SSE3, but not SSE4/5/AVX etc.
Yes, except IIRC AMD implemented SSE4.1 before SSSE3 (PSHUFB), but he claims
to see 800+ working undocumented instructions for it. So I may guess that
his tool misses lots of anyway well documented stuff.
__
wolfgang
Rick C. Hodgin
2017-10-25 17:47:17 UTC
Permalink
Post by wolfgang kern
[...]
Post by Robert Wessel
Post by wolfgang kern
your count of 800+ undocumented instructions seem to miss many SSE3/4/5,
AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A 'prefixes'. I see
only a handful (~20) 'undocumented' (alias or useless anyway) opcodes.
Didn't the OP claim to be testing an AMD X2 3800+? I think that had
SSE3, but not SSE4/5/AVX etc.
Yes, except IIRC AMD implemented SSE4.1 before SSSE3 (PSHUFB), but he claims
to see 800+ working undocumented instructions for it. So I may guess that
his tool misses lots of anyway well documented stuff.
He discusses this. Some of what he saw in a 2012 CPU was not made
public until 2015, for example. So it finds entire classes of
instructions, many of which are legitimately under development,
just not made public until they are well tested (apparently).

What worries me most of all, are the common instructions that are
undocumented and exist across manufacturers. VIA, AMD, Intel, all
have some same unknown opcode sequences, in addition to their own
unique independent ones. VIA, for example, also included some of
their crypto opcode space, but with undisclosed opcodes that were
decoded by the core, but it's not yet known what those opcodes
actually generate or do.

It's an absolutely brilliantly amazing tool Christopher Domas has
come up with. It's stunning in fact. It's prompted me to watch
other videos of his work. He's quite the genius powerhouse there.
I've added him to the list of people I'd like to work with / for.
The list includes Kostya Serebryany and Christopher Domas among a
small handful of others.

Thank you,
Rick C. Hodgin
Kerr-Mudd,John
2017-10-26 18:33:56 UTC
Permalink
Post by Rick C. Hodgin
Post by wolfgang kern
[...]
Post by Robert Wessel
Post by wolfgang kern
your count of 800+ undocumented instructions seem to miss many
SSE3/4/5, AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A
'prefixes'. I see
only a handful (~20) 'undocumented' (alias or useless anyway) opcodes.
Didn't the OP claim to be testing an AMD X2 3800+? I think that
had SSE3, but not SSE4/5/AVX etc.
Yes, except IIRC AMD implemented SSE4.1 before SSSE3 (PSHUFB), but he
claims to see 800+ working undocumented instructions for it. So I may
guess that his tool misses lots of anyway well documented stuff.
He discusses this. Some of what he saw in a 2012 CPU was not made
public until 2015, for example. So it finds entire classes of
instructions, many of which are legitimately under development,
just not made public until they are well tested (apparently).
What worries me most of all, are the common instructions that are
undocumented and exist across manufacturers. VIA, AMD, Intel, all
have some same unknown opcode sequences, in addition to their own
unique independent ones. VIA, for example, also included some of
their crypto opcode space, but with undisclosed opcodes that were
decoded by the core, but it's not yet known what those opcodes
actually generate or do.
It's an absolutely brilliantly amazing tool Christopher Domas has
come up with. It's stunning in fact. It's prompted me to watch
other videos of his work. He's quite the genius powerhouse there.
I've added him to the list of people I'd like to work with / for.
The list includes Kostya Serebryany and Christopher Domas among a
small handful of others.
Thank you,
Rick C. Hodgin
No, Thank you! for a coherent and on-topic post.
s***@hotmail.com
2017-10-27 02:52:11 UTC
Permalink
Post by wolfgang kern
[...]
Post by Robert Wessel
Post by wolfgang kern
your count of 800+ undocumented instructions seem to miss many SSE3/4/5,
AVX VEX XOP and BMI codes. Some are behind 0F38/0F3A 'prefixes'. I see
only a handful (~20) 'undocumented' (alias or useless anyway) opcodes.
Didn't the OP claim to be testing an AMD X2 3800+? I think that had
SSE3, but not SSE4/5/AVX etc.
Yes, except IIRC AMD implemented SSE4.1 before SSSE3 (PSHUFB), but he claims
to see 800+ working undocumented instructions for it. So I may guess that
his tool misses lots of anyway well documented stuff.
For now I think most of those 800.000 were probably variations of "prefetch" and "3d now" instructions.

One idea to filter this out is to "blacklist" the opcodes for these instructions in the injector.c.

For now I would be happy with just filtering out the "prefetch" opcodes.

But I am not sure how to do it correctly because it might depend on R/M mod byte or something like that ?

For now maybe blacklisting:

0f0f
or
0f0d
or
0f18

would be enough ?

I tried to run sandsifter like this... but somehow I screwed something up and it was double as slow... then I discovered that two instances were running... maybe this was bad... and somehow it screwed up the log file... but I am not sure... for now I am a bit embarrased about that... so I dare not post the second/new log file... fearing that it will be another big fail ! LOL.

So next time I run sandsifter I will pay more attention to what is actually running on linux and try and keep it as clean as possible... perhaps do a clean boot if I messed around with builds/runs/aborts/etc.

For now I am "getting the hang of linux mint a bit"... slowly... little bit by little bit... linux definetly not my favorite piece of software or cake... I find many things annoying... some things are cool though... so it needs getting used to on my part. I am also thinking of perhaps "rolling my own" based on his code but then for Windows instead.

However avoiding crashes on windows and such... not sure if I can pull it off... and I am not yet sure... if I might be putting my PC at risk... so I am a bit reluctant about that for now... don't just wanna dive in and try stuff... so I am first trying to figure out what it is I need to do to try and do it somewhat safely...

Perhaps developing this inside an "emulator"/"vmware" might be best... though I do fear/worry that this might even escape or crash the emulator in such a way that it might still affect the host... but hopefully less severe... then again such a big app as VMWare... if it gets corrupted... who knows... it might even be worse... haha... that would be funny.

Seeing the cool read/write and totally no access page trick at the end of the injector code for none-no-execution systems/old system is pretty cool and perhaps I should try that trick on a 80486...

I never tried borland pascal 32 bit that much... just turbo pascal 16 bit...

I wonder if borland pascal 32 bit had some kind of funtionality to set read/write access for pages ?

If it has that... that would be really cool... then I might "whip out" my old 80486 lol...

This would leave the disassembly problem though... need a disassembler for 80486... maybe there is still some old software on that thing that did it... and otherwise maybe internet has some old disassembler software that will work in turbo pascal or borland pascal... hmmm...

Could be fun project... and could be fun to do on older hardware... just to start "safely" or something... maybe this idea is crazy... but then again... better be safe than sorry with main system ! ;) =D

For now though maybe I will continue with linux mint and python... I probably will give it another try in the weeked to see how well the blacklisting works and this time I will make sure only one injector runs... I noticed two were running because both cores were at 100%... that shouldn't happen as far as I know sandsifter wants to run on one core mostly... it does seem to have multi-core options... not yet sure about that... for now it's at least "one control thread/process" the python script... and "one injector process/thread" doing the "deed".

Bye,
Skybuck.
wolfgang kern
2017-10-27 09:10:44 UTC
Permalink
<***@hotmail.com> wrote:
...
</q>
For now I think most of those 800.000 were probably variations of "prefetch"
and "3d now" instructions.

One idea to filter this out is to "blacklist" the opcodes for these
instructions in the injector.c.

For now I would be happy with just filtering out the "prefetch" opcodes.

But I am not sure how to do it correctly because it might depend on R/M mod
byte or something like that ?

For now maybe blacklisting:

0f0f
or
0f0d
or
0f18

would be enough ?
</q>

I see only 24 valid 0f0f 3Dnow opcodes not including the three:
0f0d/0 PREFETCH,0f0d/1 PREFETCHW and the twobyte (3Dnow+SSE2 )0f0e FEMMS.

0f18/0,m PREFETCHNTA ;they belong to SSE2 (documented since long)
0f18/1,m PREFETCHT0
0f18/2,m PREFETCHT1
0f18/3,m PREFETCHT2
0f18/4..7,m are reported as "Hint-Nops" or (K10M VEX/MVEX) instructions.
0f18C0+reg reports here "EXC6, memory only"

all prefetch instructions got only one (memory) operand, so the reg-bits
might be a dont care in 3Dnow, but not for 0F18.

Even when I count all prefix and addressing variants, there must be more lost
groups within your 800.

[about your problem...]

A full featured emulation of a modern x86 CPU with unprotected access and
decent debugger may be somehow hard to find anyway.

Test on unknow opcodes will be impossible in restricted environments like
windoze or loonix. You'll need a clean 'unprotected' OS/debug combination
to check on. And even within such a tool like mine it may crash if memory
which contain 'hot code/data' become altered.
__
wolfgang
s***@hotmail.com
2017-10-27 22:05:50 UTC
Permalink
Post by s***@hotmail.com
...
</q>
For now I think most of those 800.000 were probably variations of "prefetch"
and "3d now" instructions.
One idea to filter this out is to "blacklist" the opcodes for these
instructions in the injector.c.
For now I would be happy with just filtering out the "prefetch" opcodes.
But I am not sure how to do it correctly because it might depend on R/M mod
byte or something like that ?
0f0f
or
0f0d
or
0f18
would be enough ?
</q>
0f0d/0 PREFETCH,0f0d/1 PREFETCHW and the twobyte (3Dnow+SSE2 )0f0e FEMMS.
0f18/0,m PREFETCHNTA ;they belong to SSE2 (documented since long)
0f18/1,m PREFETCHT0
0f18 I guess that is the opcode. /digit seems to refer to ModR/m extension.

Not sure what ,m means ? probably memory address.

So another idea could be for me to read up on how to decode x86 and amd64 instructions and then just simply write a "prefetch" and/or "3d now" decoder... and "sift" through the log I already captured as to keep in line with the nature of the "SandSIFTER" lol... multiple sifts ! ;) =D

This could be fun and save me a hell lot of trouble ! ;) :) Until capstone is fixed or I have some more time to write my own.

For now I simply want to figure out what else is in there ! :)

For now I will have to hope/assume that sandsifter worked correctly... but I somehow doubt it... there might be bugs in that software with the search algorithm... but it's good enough for now ! ;) :)
Post by s***@hotmail.com
0f18/2,m PREFETCHT1
0f18/3,m PREFETCHT2
0f18/4..7,m are reported as "Hint-Nops" or (K10M VEX/MVEX) instructions.
0f18C0+reg reports here "EXC6, memory only"
all prefetch instructions got only one (memory) operand, so the reg-bits
might be a dont care in 3Dnow, but not for 0F18.
Even when I count all prefix and addressing variants, there must be more lost
groups within your 800.
[about your problem...]
A full featured emulation of a modern x86 CPU with unprotected access and
decent debugger may be somehow hard to find anyway.
Test on unknow opcodes will be impossible in restricted environments like
windoze or loonix.
Why ? So far sandsifter/injector seems to be able to do it.
Post by s***@hotmail.com
You'll need a clean 'unprotected' OS/debug combination
to check on. And even within such a tool like mine it may crash if memory
which contain 'hot code/data' become altered.
SandSifter/Injector took all kinds of precautions to try and avoid that :)

Bye,
Skybuck.
s***@hotmail.com
2017-10-27 22:47:15 UTC
Permalink
I think I am in llllllllluuuuuuuuuuuck !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

BOOM BABY !

DELPHI POWERRRRRRRRR ! =D

https://github.com/MahdiSafsafi/DebugEngine

This folder has a subfolder called: Common/UnivDisasm

As far as I can tell it is fricking awesome ! It has everything I ever heard of.... mmx, prefetch, 3dnow, avx whatever...

Gonna check the inc files to see if it's actually there just in case ! ;)

Seems legit ! LOL.

1 year old code....

I feel bleshed...

I don't wanna be arrogant or anything ! LOLOLOLOLOLOLOL.

But I think this code will run circles around capstone when it comes to x86 and x64 decoding... or let's just call it 32 and 64 bit decoding ! LOLOLOL.

And best of all..

Skkkkyyybbbuuuccck

^^^^ Very experienced Delphi coder/programmer ^^^^^ =======DDDDD

SWWWEEEET ! ;) =D

This "custom-sifter" project seems feasible... high chance of a success and will probably be done in a snap ! ;) =D =D =D =D =D =D =D =D =D =D =D

Most of all it gonna be a little bit of fun !

Drawback is... I learn less about reading the docs... but hell... screw the docs as usual... Delphi got me covered ! LOLOLOL.

No I might read docs maybe... if needed for now...

Skybuck-Dolphin-Diving-Back-Into-The-Brave-World-And-Awesome-Quality-World-Of-DDDDDDDDDDDELLLLLPPPHIIII !

And this is just one of them... there are others out there tooo... also found on written in C:

http://beatrix2004.free.fr/BeaEngine/index1.php

But the Delphi one above... seems more up my alley ! ;)

Bless you all... and Skybuck over and out for now.

Drops the mic on the floor ! LOL.

Bye,
Skybuck ! ;) =D
s***@hotmail.com
2017-10-27 22:54:56 UTC
Permalink
This seems to be the homepage of this UnivDisasm:

https://github.com/MahdiSafsafi/UnivDisasm

Reading it now... most interesting... hope it doesn't contain too many bugs ! ;) :)

Bye,
Skybuck.
s***@hotmail.com
2017-10-27 22:59:56 UTC
Permalink
LOL,

I am starting to wonder if my DreamPC from 2006 has been hit by some kind of malware...

The FAN on the northbridge chip (?) keeps malfunctioning from time to time... have cleaned it like 10 times already ! LOL.

Probably just dirt getting in there...

But the thought of STUXNET-like virus being in there did cross my mind just now ! LOLOLOLOL.

Gosh... suppose somebody would write a virus/worm to shutdown all PC fans... wow... then the world would be in trouble ! ;) :)

Bye,
Skybuck.
s***@hotmail.com
2017-10-27 23:03:55 UTC
Permalink
It's ok... now it's spinning fine again ! HAHA :)

Maybe because the FAN spins so fast... hihi... the virus/worm believe it already did enough damage ! LOL.

Bye,
Skybuck.

P.S.: Ok gotta stop the fun... it's polluting this thread... such a polluter I am... but now as worse as wolfgang with his big fat ass truck ??? LOL :)

Why is a trucker programming ? ;) :) I forgot... whhhhh---oopsie
s***@hotmail.com
2017-10-27 23:11:03 UTC
Permalink
Ok one last thing, I think the real reason why this fan misbehaves sometimes... is because of the "load" on the system... with many browser tabs open in firefox...

Somehow this fan starts "oscilating"... it goes faster/slower/faster/slower... just slightly and then suddenly it gets into this "fibrilating" state...

Like a heart-attack or something....

Pretty crazy... that PCs can now have heart attacks ! ;) :) Just like human beings !

My PC might soon need a "defibrilator !" HaHa...

We humans and PCs are apperently not that different...I already knew that... but this... wow... just wow... add it to your list of "simularities" ! LOL.

What's next ? ;) Monitors going blurry and needing glasses ? ;)

Rollators ? Hmmm ? Hmmmmmmm.....

Diepers ? :)

Actually yeah... water leakage for water cooled systems... yes PCs can leak ! like grandpa and grandma ! HA-HA-HA ! =D

And they poop tooo... I removed a lot of fluffy poop from the blades today ! Yikes... I thought that could catch fire... and sure enough... a few hours later there was a fire outside ! Unbelievable coincidences the universe is in !

I wrote about this some time ago... apperently it's a cyclic thing... hmmm might have to start paying notice to this...

Perhaps I have discovered something about the "under lieing" universe...

The random generator... has a "coincedence" period... sometimes it will enter this period... and all kinds of stuff start to happen like a "coincedence".

For example... today I had my hear cut... I questioned the cutter... a girl... I asked... what is your name when she was done ?

Guess what she said...

"Isa !"

And I immediately thouth... gjezus... sounds like "api"... I couldn't immediately put my finger on it ! (LOL. Damn she was hot ! But no no no finger action for me ! LOL.)

But later today... I did put my finger on it... figurely speaking...

ISA means Instruction Set Architecture !

How's that for a mothafokking coincidence ?!

True story ! Sigh.

Bye,
Skybuck.
Rod Pemberton
2017-10-28 18:57:23 UTC
Permalink
On Fri, 27 Oct 2017 16:11:03 -0700 (PDT)
Post by s***@hotmail.com
Somehow this fan starts "oscilating"... it goes
faster/slower/faster/slower... just slightly and then suddenly it
gets into this "fibrilating" state...
That could be a BIOS setting. I turn off one setting in my BIOS which
automatically adjusts the fan speed based on temperature. Sometimes the
fan will start speeding up/slowing down, even though the setting is
still reported as disabled in the BIOS settings. So, periodically, I
have to toggle it on in BIOS, save the BIOS, reboot, then toggle it off
in the BIOS, save the BIOS, reboot. I think there is another BIOS
setting for software control of the fan too.
Post by s***@hotmail.com
Like a heart-attack or something....
I've had more than a few computer fans that have started squealing,
just like a pig. More than one of them was fixed by simply removing any
dust buildup. A small screwdriver and flashlight are useful if you
don't have a miniature vacuum cleaner.

If the fan chirps mildly, it's probably just slightly out of balance,
i.e., blades wobbling and hitting the casing. I usually just touch a
finger on the spinning center of the fan and push down very slightly,
but I don't stop the fan spinning as that could cause electrical
issues. This should re-balance the blades so they stop chirping. Also,
be careful and don't hit the spinning blades with your finger as you
can take off part of your a finger nail ... BTDT, unintentionally.
The fan motors are not that powerful, but the blades have a high rate
of speed and they hurt about as bad as hitting your finger with a
hammer.

For one other consistently squealing fan, I had to remove a sticker on
the blades, and then a screw to get to the bearings. The factory didn't
lubricate the bearings. Or, whatever they did lubricate them with
dissipated over time. If this is the problem, you may need some WD-40
to clean the bearings, and some Liquid Wrench silicone spray for
lubrication. These are available at most auto parts stores in the U.S.,
and probably most hardware stores too. If you can't find silicone
lubricant, you might be able to find spray Teflon. I repacked some
bicycle main bearings with Teflon grease decades ago, and they still
spin like there is no friction at all.


Rod Pemberton
--
Why are Hillary's living donors pedophiles and rapists? Bill Clinton,
Harvey Weinstein, Anthony Wiener, Jeffrey Epstein, ...
s***@hotmail.com
2017-10-29 01:48:36 UTC
Permalink
It's a tiny little fan on the motherboard.

My other hypothesis is that fumes from car or busses get inside the fan... as slightly "dark matter" LOL.

Even dark matter is now inside my computer ! =DDD

I have another follow-up explanation for this:

1. Either I vent my place more and thereby allowing more black dust in.
2. Or the fan at the back of the building that stops spinning somewhat filters less air.
or my favorite:
3. The new busses in my city contain "scamming-software" and actually pollute much more than the old busses !

I'm betting on 3 for now ! ;) :) though 1 might be the case as well.

Lastly:

4. My pc is just damn dirty and keeps filling up the fan with dust... I did remove some big fat dust from the case fan blades though.. maybe that will help somewhat...

Oddly enough the center fan had huge ammounts of dust/fluf on it... while the bottom fan very little... and the fan at the top also not that much...

I wonder why the fan in the middle has so much.....

I guess it's simply sucking up the most fabric from my cloths.

Cause my upper body is right in the middle of it.... sort of...

Maybe the bottom fan has some airflow blocked by the mouse and my arm in the way...

Or it has something to do with the side hole near the PC it is at the same height...

Perhaps the front center fan... is sucking back in durst particles expelled from that side hole ?

Yes.... that must be it... I suddenly remember I turned the CPU cooler/fan upside down... so it's blowing air out of my PC instead of in...

I totally forgot about that...

This could also explain why the table always has so much dust on it... though that could also simply be gravity at work...

Hmm... very interesting hypothesis..

It's probably both... my cloth... but mostly that side panel hole... expelling dust particles.... that has got to be the case of it !

Hmmm....

I bet there is a circular air flow thing going on.... hmmmm...

I will take this into consideration in my next PC built !

Guess this posting was usefull after all ! ;) :)

Bye,
Skybuck.
wolfgang kern
2017-10-29 08:07:42 UTC
Permalink
Rod Pemberton posted:

[about dirty CPU fans...]
you're right, I have to clean out my PCs once a year and I made an addon to my
vacuum-cleaner with a 1/2 inch plastic tube and a fitting cover-tip of a
ball-pen. This way I can easy suck all these tiny slots behind the CPU-fan,
but better dont tell this your girlfriend :)
__
wolfgang
wolfgang kern
2017-10-28 08:45:49 UTC
Permalink
<***@hotmail.com> wrote:
...
Post by wolfgang kern
0f0d/0 PREFETCH,0f0d/1 PREFETCHW and the twobyte (3Dnow+SSE2 )0f0e FEMMS.
0f18/0,m PREFETCHNTA ;they belong to SSE2 (documented since long)
0f18/1,m PREFETCHT0
|0f18 I guess that is the opcode. /digit seems to refer to ModR/m extension.

Yes,

0f18/0 mean 0f18 00..07
0f18/1 mean 0f18 08..0F
an so on

|Not sure what ,m means ? probably memory address.

Yes memory only, no register operands allowed.

|For now I simply want to figure out what else is in there ! :)

Me too would like to see at least one 'undocumented' which does something
that's not a NOP or a redundant alias. Haven't seen a single one so far
on x86 except SALC decades ago.
Post by wolfgang kern
[about your problem...]
A full featured emulation of a modern x86 CPU with unprotected access and
decent debugger may be somehow hard to find anyway.
Test on unknow opcodes will be impossible in restricted environments like
windoze or loonix.
|Why ? So far sandsifter/injector seems to be able to do it.
Post by wolfgang kern
You'll need a clean 'unprotected' OS/debug combination to check on.
And even within such a tool like mine it may crash if memory which
contain 'hot code/data' become altered.
|SandSifter/Injector took all kinds of precautions to try and avoid that :)

If true then I assume that it works in an emulated environment on an emulated
CPU (this might use a lot of guessing).

My way is somehow less detouring: I enter the unknown opcode into a free
memory area, pad it with 0x90 NOPS, set all registers to known values and
then hit the single step key. So I see either an exception or upgraded
registers incl. EIP and flags, and then I check if any memory locations
pointed to by 0x90.. or by my initial register settings were altered.
I once used this in a loop to detect all present MSRs, even non present
ones may not raise an exeption.
__
wolfgang
s***@hotmail.com
2017-10-22 20:25:09 UTC
Permalink
Hi,

I hope you have the following newsgroup in case you are highly interested in knowing every last detail and every thought I have on this subject matter:

https://groups.google.com/forum/#!forum/alt.comp.hardware.pc-homebuilt

There I have written some detailed postings.

In the other newsgroups I will constrain myself to the most important matter/summary of my activities, findings and productions for your usage.

The most important information I want to share with you is the following:

1. I was successfull in running SandSifter software with Linux Mint 18.2 booteable DVD, downloaded ISO from the internet and undocumented instructions have been found for AMD X2 3800+ Dual Core processor.

2. All files are available on my webdrive:

www.skybuck.org/SandSifter/

Explore the "unzipped folder" to see what it's all about.

3. I have written two tutorials how you can also run this software on your computer in case you have a DVD drive and DVD disc to burn this software onto.
One manual tutorial and one automatic tutorial. The automatic tutorial is the easiest one which I will post here, the automatic tutorial includes a run.sh script which I will also post here, this is to help you run this software on your machine, at the end of this posting I will discuss any possible risks to doing so in case you are worried.

Automatic tutorial:

Step 1. Download Linux Mint ISO (Successfully tested on Linux Mint 18.2 Sonya)

https://www.linuxmint.com/

Step 2. Burn Linux Mint ISO to DVD (Windows 7: Right click on file and choose burn to disc).

Step 3. Boot Linux Mint ISO from DVD (Restart computer, if needed go into bios and change boot order, or press F8 to bring up boot menu or something like that)

Step 4. Start FireFox Web Browser

Step 5. Download SandSifter software and extract to a folder.

https://github.com/xoreaxeaxeax/sandsifter

(Click "clone or download", then click "download zip", then click "open with archive manager", then click "extract" (top left icon), click "other locations", choose a harddisk or other storage
medium which is persistent, click on the storage medium, click create new folder (top right icon), name for folder could be "test", click "extract", click "show the files")

Enter the folder "sandsifter-master" by left clicking on it.

Step 6. Download Skybuck's Flying run.sh script file

Download and save the "run.sh" script file to/inside the "sandsifter-master" folder.

http://www.skybuck.org/SandSifter/unzipped/run.sh

Step 7. Open terminal window and resize it to make it bigger

Right click in the empty space and choose "open in terminal"

A window and a prompt/blinking cursor should now come up looking similar to:

***@mint /media/mint/Windows 7 System (New)/test/sandsifter-master $

Make the window bigger so that the summarize script at the end doesn't crash !

Drag and Drop the window at the bottom right corner to make it bigger (Hold the left mouse button to drag and make it bigger then let mouse button go)

Step 8. Run Skybuck's Flying Bash Script to install software and run SandSifter

type the following command:

bash ./run.sh

Step 9. Guide the software installation and upgrade process

Sometimes it will ask if you want to continue ? Press the Y key.

Once it's done installing SandSifter will automatically run and finally a summary will be created.

Step 10. Wait for the analysis to complete

Once you see instructions scrolling/flying over the screen go take a sleep and wait many hours until it is completely done.

Once it is done it will show something like: "May the Force be with you ! Always !" then you know the script is done !

Step 11. Do not open the log files !

The log files (in data folder) may be to big for the Linux Mint 18.2 text and office editors to handle ! This will probably crash/hang the system !

Step 12. Go into the data folder and send the files to the e-mail address:

***@gmail.com


The run.sh script:

echo "Step 1. Install standard C library software"
sudo apt-get install libc6-dev

echo "Step 2. Install python pip"
sudo apt install python-pip

echo "Step 3. Update python pip"
sudo pip install --upgrade pip

echo "Step 4. Install setuptools"
sudo pip install setuptools

echo "Step 5. Install capstone binaries"
sudo apt-get install libcapstone3

echo "Step 6. Install capstone dev source"
sudo apt-get install libcapstone-dev

echo "Step 7. Install capstone python bindings (this will take a while)"
sudo pip install capstone

echo "Step 8. Make sandsifter"
make

echo "Step 9. Run sandsifter"
sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t

echo "Step 10. Summarize"
./summarize.py data/log

echo ""
echo "Bash script"
echo "Version 0.01 created on 22 october 2017 by Skybuck Flying"
echo "To Install, Make, Run, Summarize SandSifter Software and Software Dependencies"
echo "Successfully tested on Linux Mint 18.2 Sonya on AMD Dual Core X2 3800+ processor"
echo "May the Force be with you ! Always ! =D"
echo "Have fun analyzing undocumented instructions !!!!"
echo "E-mail results to or contact: ***@gmail.com"
echo "^^^ !!! Author of SandSifter Software and interested in log files !!! ^^^"
echo ""

For now I will not discuss the collected data, this will have to be further analyzed, however I will say that the collected data is in this folder:

http://www.skybuck.org/SandSifter/unzipped/data/

The log file contains discovered undocumented instruction byte code sequences for further investigation.

(Lastly I will try and collect the messages I write on this subject matter in the messages folder so you don't have to scavenge the usenet/web for all info;) a bit tricky but I will try at least :))

Bye,
Skybuck.
s***@hotmail.com
2017-10-23 23:21:53 UTC
Permalink
Idea of this software is basically:

Generate random bytes and feed them to processor.

Observe result of processor if good or bad (error codes).

If good check docs.
If bad adjust and retry.

Somebody wrote a nice short explanation of what SandSifter does to give you an idea (it's a new algorithm to find undocumented instructions fast !):

It's guessing possible X86 instructions by exploiting the Instruction Decoder via the (PF) Page Fault result code. Effectively splitting an instruction across two pages and only having one page of it executable. When the decoder fetches the instruction it notices that it's incomplete, attempts to fetch the next part that is on a new non-executable page. The decoder then throws a page fault since it's not executable. So it moves the entire instruction one to the left and tries again with various combinations until it doesn't get a page fault at which point it executes it.

And thus it attempts to 'tunnel' through every possible instruction. That's the general very simplified explanation.

Bye,
Skybuck.
s***@hotmail.com
2017-10-29 21:27:57 UTC
Permalink
I am going to issue a warning about all of this software:

SandSifter, Linux Mint 18.2 and install-apt and for windows: git

For now I suspect running two instances of SandSifter at same time on Linux Mint 18.2 caused file system corruption as can be seen on these three screenshots also checkdisk log file is included in web folder:

http://www.skybuck.org/SandSifter/unzipped/risks/

Possible causes of file system corruptions:

1. Running two instances of SandSifter + Linux Mint 18.2
2. Git on Windows
3. Perhaps a problem was already with file system.
4. BIOS Corruption in recent past.
5. Spark when other person connected laptop to power output... there was a spark.
6. FireFox corruption while browsing or extracting files !
Google detected cooky corruption... so FireFox is also a prime suspect !

Be carefull !

I still have to investigate checkdisk log further though ! ;)

Bye,
Skybuck.
s***@hotmail.com
2017-10-29 22:13:33 UTC
Permalink
I do remember a very rare and extreme Windows 7 system hang not so long ago.

It was probably caused by having to many FireFox tabs open... and somehow a website/FireFox managed to "hang windows 7".

Windows 7 might have been busy trying to write data to the harddisk and somehow during that process it hang.

After waiting a bit I had no choice but to press the "reset" button.

I think this is what might have caused this file system corruption.

A combination of "FireFox memory hogging" and "Windows 7 writing to disk", possibly FireFox cache or cookie related.

Bye,
Skybuck.

Loading...