Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1661 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1213. Min Range: -2.1213

Max Act: 2.1213. Min Act: -0.1700

Data Index: 6815176 (Open Web Text)

Max Activating Token Index: 1009

Text #1

Max Range: 2.1213. Min Range: -2.1213

Max Act: 2.0803. Min Act: -0.1700

Data Index: 8734231 (Open Web Text)

Max Activating Token Index: 487

Text #2

Max Range: 2.1213. Min Range: -2.1213

Max Act: 2.0323. Min Act: -0.1700

Data Index: 6733803 (Open Web Text)

Max Activating Token Index: 877

Text #3

Max Range: 2.1213. Min Range: -2.1213

Max Act: 2.0220. Min Act: -0.1700

Data Index: 8478759 (Open Web Text)

Max Activating Token Index: 793

Text #4

Max Range: 2.1213. Min Range: -2.1213

Max Act: 2.0159. Min Act: -0.1700

Data Index: 2011548 (Open Web Text)

Max Activating Token Index: 513

Text #5

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9787. Min Act: -0.1700

Data Index: 880610 (Open Web Text)

Max Activating Token Index: 300

Text #6

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9827. Min Act: -0.1700

Data Index: 6035489 (Open Web Text)

Max Activating Token Index: 865

Text #7

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9686. Min Act: -0.1700

Data Index: 6291513 (Open Web Text)

Max Activating Token Index: 295

Text #8

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9477. Min Act: -0.1700

Data Index: 2652624 (Open Web Text)

Max Activating Token Index: 357

Text #9

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9381. Min Act: -0.1700

Data Index: 2558961 (Open Web Text)

Max Activating Token Index: 433

Text #10

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9366. Min Act: -0.1700

Data Index: 1072702 (Open Web Text)

Max Activating Token Index: 147

Text #11

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9355. Min Act: -0.1700

Data Index: 3388214 (Open Web Text)

Max Activating Token Index: 981

Text #12

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9145. Min Act: -0.1700

Data Index: 5200876 (Open Web Text)

Max Activating Token Index: 640

Text #13

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9279. Min Act: -0.1700

Data Index: 5759405 (Open Web Text)

Max Activating Token Index: 763

Text #14

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9255. Min Act: -0.1700

Data Index: 4389551 (Open Web Text)

Max Activating Token Index: 360

Text #15

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9225. Min Act: -0.1700

Data Index: 1846768 (Open Web Text)

Max Activating Token Index: 783

Text #16

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9116. Min Act: -0.1700

Data Index: 5278055 (Open Web Text)

Max Activating Token Index: 451

Text #17

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9187. Min Act: -0.1700

Data Index: 2760230 (Open Web Text)

Max Activating Token Index: 989

Text #18

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.9076. Min Act: -0.1700

Data Index: 6847894 (Open Web Text)

Max Activating Token Index: 143

Text #19

Max Range: 2.1213. Min Range: -2.1213

Max Act: 1.8907. Min Act: -0.1700

Data Index: 5218135 (Open Web Text)

Max Activating Token Index: 199