Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2171 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.8334. Min Act: -0.1700

Data Index: 2778108 (Open Web Text)

Max Activating Token Index: 69

Text #1

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.7236. Min Act: -0.1700

Data Index: 1902070 (Open Web Text)

Max Activating Token Index: 143

Text #2

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.7036. Min Act: -0.1699

Data Index: 7934108 (Open Web Text)

Max Activating Token Index: 360

Text #3

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.7046. Min Act: -0.1699

Data Index: 6214484 (Open Web Text)

Max Activating Token Index: 635

Text #4

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6932. Min Act: -0.1700

Data Index: 4676707 (Open Web Text)

Max Activating Token Index: 858

Text #5

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6992. Min Act: -0.1700

Data Index: 6748119 (Open Web Text)

Max Activating Token Index: 205

Text #6

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6953. Min Act: -0.1675

Data Index: 7226057 (Open Web Text)

Max Activating Token Index: 334

Text #7

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6877. Min Act: -0.1700

Data Index: 861703 (Open Web Text)

Max Activating Token Index: 143

Text #8

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6694. Min Act: -0.1699

Data Index: 4527763 (Open Web Text)

Max Activating Token Index: 85

Text #9

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6765. Min Act: -0.1700

Data Index: 5469613 (Open Web Text)

Max Activating Token Index: 156

Text #10

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6834. Min Act: -0.1700

Data Index: 2216202 (Open Web Text)

Max Activating Token Index: 484

Text #11

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6735. Min Act: -0.1700

Data Index: 7369963 (Open Web Text)

Max Activating Token Index: 72

Text #12

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6559. Min Act: -0.1690

Data Index: 634971 (Open Web Text)

Max Activating Token Index: 143

Text #13

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6487. Min Act: -0.1700

Data Index: 2907956 (Open Web Text)

Max Activating Token Index: 120

Text #14

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6445. Min Act: -0.1700

Data Index: 5419092 (Open Web Text)

Max Activating Token Index: 236

Text #15

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6420. Min Act: -0.1700

Data Index: 3449997 (Open Web Text)

Max Activating Token Index: 90

Text #16

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6442. Min Act: -0.1680

Data Index: 7819627 (Open Web Text)

Max Activating Token Index: 425

Text #17

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6343. Min Act: -0.1700

Data Index: 5469626 (Open Web Text)

Max Activating Token Index: 332

Text #18

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6364. Min Act: -0.1700

Data Index: 6496689 (Open Web Text)

Max Activating Token Index: 143

Text #19

Max Range: 2.8334. Min Range: -2.8334

Max Act: 2.6162. Min Act: -0.1699

Data Index: 866085 (Open Web Text)

Max Activating Token Index: 86