Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2049 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5921. Min Act: -0.1700

Data Index: 2818002 (Open Web Text)

Max Activating Token Index: 802

Text #1

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5984. Min Act: -0.1700

Data Index: 4637900 (Open Web Text)

Max Activating Token Index: 828

Text #2

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.6019. Min Act: -0.1700

Data Index: 269826 (Open Web Text)

Max Activating Token Index: 679

Text #3

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5683. Min Act: -0.1700

Data Index: 191242 (Open Web Text)

Max Activating Token Index: 765

Text #4

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5830. Min Act: -0.1700

Data Index: 4309111 (Open Web Text)

Max Activating Token Index: 768

Text #5

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5734. Min Act: -0.1700

Data Index: 7436074 (Open Web Text)

Max Activating Token Index: 439

Text #6

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5630. Min Act: -0.1700

Data Index: 8207676 (Open Web Text)

Max Activating Token Index: 696

Text #7

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5636. Min Act: -0.1700

Data Index: 7128685 (Open Web Text)

Max Activating Token Index: 214

Text #8

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5735. Min Act: -0.1700

Data Index: 5420142 (Open Web Text)

Max Activating Token Index: 828

Text #9

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5630. Min Act: -0.1700

Data Index: 2593324 (Open Web Text)

Max Activating Token Index: 246

Text #10

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5539. Min Act: -0.1699

Data Index: 6584232 (Open Web Text)

Max Activating Token Index: 164

Text #11

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5504. Min Act: -0.1700

Data Index: 4199908 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5366. Min Act: -0.1700

Data Index: 337390 (Open Web Text)

Max Activating Token Index: 511

Text #13

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5514. Min Act: -0.1699

Data Index: 6044197 (Open Web Text)

Max Activating Token Index: 127

Text #14

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5467. Min Act: -0.1700

Data Index: 4676841 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5429. Min Act: -0.1700

Data Index: 2905891 (Open Web Text)

Max Activating Token Index: 471

Text #16

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5292. Min Act: -0.1700

Data Index: 5668354 (Open Web Text)

Max Activating Token Index: 223

Text #17

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5376. Min Act: -0.1700

Data Index: 2136334 (Open Web Text)

Max Activating Token Index: 116

Text #18

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5434. Min Act: -0.1700

Data Index: 1716681 (Open Web Text)

Max Activating Token Index: 593

Text #19

Max Range: 1.6019. Min Range: -1.6019

Max Act: 1.5396. Min Act: -0.1700

Data Index: 7812155 (Open Web Text)

Max Activating Token Index: 231