Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2497 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.7066. Min Act: -0.1700

Data Index: 7405641 (Open Web Text)

Max Activating Token Index: 415

Text #1

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6613. Min Act: -0.1698

Data Index: 130005 (Open Web Text)

Max Activating Token Index: 210

Text #2

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6349. Min Act: -0.1700

Data Index: 950383 (Open Web Text)

Max Activating Token Index: 186

Text #3

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6360. Min Act: -0.1698

Data Index: 4753712 (Open Web Text)

Max Activating Token Index: 137

Text #4

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6221. Min Act: -0.1700

Data Index: 3104634 (Open Web Text)

Max Activating Token Index: 321

Text #5

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6176. Min Act: -0.1700

Data Index: 7449720 (Open Web Text)

Max Activating Token Index: 176

Text #6

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6012. Min Act: -0.1700

Data Index: 5420890 (Open Web Text)

Max Activating Token Index: 675

Text #7

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6124. Min Act: -0.1699

Data Index: 8443934 (Open Web Text)

Max Activating Token Index: 363

Text #8

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5977. Min Act: -0.1700

Data Index: 224848 (Open Web Text)

Max Activating Token Index: 38

Text #9

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5907. Min Act: -0.1700

Data Index: 5383029 (Open Web Text)

Max Activating Token Index: 384

Text #10

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5906. Min Act: -0.1700

Data Index: 3806168 (Open Web Text)

Max Activating Token Index: 588

Text #11

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5912. Min Act: -0.1700

Data Index: 2424215 (Open Web Text)

Max Activating Token Index: 114

Text #12

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5835. Min Act: -0.1700

Data Index: 6468504 (Open Web Text)

Max Activating Token Index: 364

Text #13

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5912. Min Act: -0.1700

Data Index: 2341214 (Open Web Text)

Max Activating Token Index: 279

Text #14

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5797. Min Act: -0.1700

Data Index: 6352519 (Open Web Text)

Max Activating Token Index: 508

Text #15

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5680. Min Act: -0.1698

Data Index: 4938781 (Open Web Text)

Max Activating Token Index: 175

Text #16

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5681. Min Act: -0.1700

Data Index: 2102642 (Open Web Text)

Max Activating Token Index: 153

Text #17

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5638. Min Act: -0.1690

Data Index: 4938779 (Open Web Text)

Max Activating Token Index: 297

Text #18

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5557. Min Act: -0.1700

Data Index: 3606199 (Open Web Text)

Max Activating Token Index: 827

Text #19

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.5604. Min Act: -0.1700

Data Index: 1026441 (Open Web Text)

Max Activating Token Index: 135