Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 506 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.2127. Min Act: -0.1700

Data Index: 584783 (Open Web Text)

Max Activating Token Index: 671

Text #1

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.1279. Min Act: -0.1700

Data Index: 685313 (Open Web Text)

Max Activating Token Index: 320

Text #2

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0771. Min Act: -0.1700

Data Index: 6071676 (Open Web Text)

Max Activating Token Index: 630

Text #3

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0854. Min Act: -0.1700

Data Index: 1662818 (Open Web Text)

Max Activating Token Index: 979

Text #4

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0373. Min Act: -0.1700

Data Index: 6034647 (Open Web Text)

Max Activating Token Index: 598

Text #5

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0501. Min Act: -0.1700

Data Index: 3791270 (Open Web Text)

Max Activating Token Index: 946

Text #6

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0066. Min Act: -0.1700

Data Index: 2492410 (Open Web Text)

Max Activating Token Index: 870

Text #7

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0224. Min Act: -0.1700

Data Index: 5651634 (Open Web Text)

Max Activating Token Index: 480

Text #8

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0353. Min Act: -0.1700

Data Index: 6541509 (Open Web Text)

Max Activating Token Index: 306

Text #9

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0254. Min Act: -0.1700

Data Index: 1253684 (Open Web Text)

Max Activating Token Index: 624

Text #10

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0251. Min Act: -0.1700

Data Index: 7305077 (Open Web Text)

Max Activating Token Index: 562

Text #11

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9667. Min Act: -0.1700

Data Index: 3819946 (Open Web Text)

Max Activating Token Index: 730

Text #12

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0012. Min Act: -0.1700

Data Index: 4968128 (Open Web Text)

Max Activating Token Index: 886

Text #13

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9968. Min Act: -0.1700

Data Index: 8586106 (Open Web Text)

Max Activating Token Index: 783

Text #14

Max Range: 4.2127. Min Range: -4.2127

Max Act: 4.0108. Min Act: -0.1700

Data Index: 8172493 (Open Web Text)

Max Activating Token Index: 636

Text #15

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9770. Min Act: -0.1700

Data Index: 7947466 (Open Web Text)

Max Activating Token Index: 328

Text #16

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9851. Min Act: -0.1700

Data Index: 7729478 (Open Web Text)

Max Activating Token Index: 297

Text #17

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9752. Min Act: -0.1700

Data Index: 3115434 (Open Web Text)

Max Activating Token Index: 356

Text #18

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9901. Min Act: -0.1700

Data Index: 6649938 (Open Web Text)

Max Activating Token Index: 720

Text #19

Max Range: 4.2127. Min Range: -4.2127

Max Act: 3.9896. Min Act: -0.1700

Data Index: 6366251 (Open Web Text)

Max Activating Token Index: 301