Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 794 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.1301. Min Act: -0.1513

Data Index: 3779037 (Open Web Text)

Max Activating Token Index: 18

Text #1

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0651. Min Act: -0.1202

Data Index: 2292990 (Open Web Text)

Max Activating Token Index: 77

Text #2

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0563. Min Act: -0.1555

Data Index: 126766 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0406. Min Act: -0.1127

Data Index: 8421896 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0320. Min Act: -0.1502

Data Index: 1869857 (Open Web Text)

Max Activating Token Index: 13

Text #5

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0211. Min Act: -0.1307

Data Index: 8665710 (Open Web Text)

Max Activating Token Index: 413

Text #6

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0063. Min Act: -0.1378

Data Index: 4462545 (Open Web Text)

Max Activating Token Index: 483

Text #7

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0085. Min Act: -0.1083

Data Index: 5025445 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0106. Min Act: -0.1439

Data Index: 4976621 (Open Web Text)

Max Activating Token Index: 83

Text #9

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0065. Min Act: -0.1465

Data Index: 3062043 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0043. Min Act: -0.1368

Data Index: 984774 (Open Web Text)

Max Activating Token Index: 53

Text #11

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0051. Min Act: -0.1235

Data Index: 1980312 (Open Web Text)

Max Activating Token Index: 12

Text #12

Max Range: 1.1301. Min Range: -1.1301

Max Act: 1.0065. Min Act: -0.1232

Data Index: 6526805 (Open Web Text)

Max Activating Token Index: 53

Text #13

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9880. Min Act: -0.1050

Data Index: 4607985 (Open Web Text)

Max Activating Token Index: 76

Text #14

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9754. Min Act: -0.1483

Data Index: 7075834 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9771. Min Act: -0.1050

Data Index: 707706 (Open Web Text)

Max Activating Token Index: 20

Text #16

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9743. Min Act: -0.1471

Data Index: 7082645 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9719. Min Act: -0.1612

Data Index: 7897278 (Open Web Text)

Max Activating Token Index: 380

Text #18

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9733. Min Act: -0.1388

Data Index: 6111877 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 1.1301. Min Range: -1.1301

Max Act: 0.9726. Min Act: -0.1506

Data Index: 541267 (Open Web Text)

Max Activating Token Index: 112