Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2188 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0843. Min Act: -0.1700

Data Index: 3112429 (Open Web Text)

Max Activating Token Index: 300

Text #1

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0642. Min Act: -0.1700

Data Index: 5469186 (Open Web Text)

Max Activating Token Index: 769

Text #2

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0655. Min Act: -0.1700

Data Index: 4705593 (Open Web Text)

Max Activating Token Index: 717

Text #3

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0565. Min Act: -0.1700

Data Index: 5682712 (Open Web Text)

Max Activating Token Index: 354

Text #4

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0165. Min Act: -0.1700

Data Index: 6684506 (Open Web Text)

Max Activating Token Index: 730

Text #5

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0302. Min Act: -0.1700

Data Index: 3757869 (Open Web Text)

Max Activating Token Index: 659

Text #6

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0288. Min Act: -0.1700

Data Index: 4952381 (Open Web Text)

Max Activating Token Index: 84

Text #7

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0103. Min Act: -0.1700

Data Index: 4008051 (Open Web Text)

Max Activating Token Index: 924

Text #8

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9952. Min Act: -0.1700

Data Index: 3902874 (Open Web Text)

Max Activating Token Index: 169

Text #9

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9970. Min Act: -0.1700

Data Index: 6925503 (Open Web Text)

Max Activating Token Index: 698

Text #10

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0110. Min Act: -0.1700

Data Index: 4187121 (Open Web Text)

Max Activating Token Index: 897

Text #11

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9964. Min Act: -0.1700

Data Index: 2642518 (Open Web Text)

Max Activating Token Index: 743

Text #12

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9712. Min Act: -0.1700

Data Index: 6248354 (Open Web Text)

Max Activating Token Index: 723

Text #13

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9993. Min Act: -0.1700

Data Index: 3367228 (Open Web Text)

Max Activating Token Index: 505

Text #14

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0009. Min Act: -0.1700

Data Index: 5804834 (Open Web Text)

Max Activating Token Index: 746

Text #15

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9861. Min Act: -0.1700

Data Index: 5311616 (Open Web Text)

Max Activating Token Index: 144

Text #16

Max Range: 4.0843. Min Range: -4.0843

Max Act: 4.0036. Min Act: -0.1700

Data Index: 1318992 (Open Web Text)

Max Activating Token Index: 363

Text #17

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9323. Min Act: -0.1700

Data Index: 1116083 (Open Web Text)

Max Activating Token Index: 444

Text #18

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9668. Min Act: -0.1700

Data Index: 4865698 (Open Web Text)

Max Activating Token Index: 920

Text #19

Max Range: 4.0843. Min Range: -4.0843

Max Act: 3.9587. Min Act: -0.1700

Data Index: 365262 (Open Web Text)

Max Activating Token Index: 46