Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2812 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.8241. Min Act: -0.1700

Data Index: 2141320 (Open Web Text)

Max Activating Token Index: 971

Text #1

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.7083. Min Act: -0.1700

Data Index: 5601893 (Open Web Text)

Max Activating Token Index: 400

Text #2

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.5633. Min Act: -0.1700

Data Index: 6776003 (Open Web Text)

Max Activating Token Index: 293

Text #3

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.5313. Min Act: -0.1700

Data Index: 8050112 (Open Web Text)

Max Activating Token Index: 720

Text #4

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.5429. Min Act: -0.1700

Data Index: 1446534 (Open Web Text)

Max Activating Token Index: 579

Text #5

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.5384. Min Act: -0.1700

Data Index: 4039635 (Open Web Text)

Max Activating Token Index: 793

Text #6

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.5018. Min Act: -0.1700

Data Index: 2062623 (Open Web Text)

Max Activating Token Index: 729

Text #7

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4965. Min Act: -0.1700

Data Index: 8307560 (Open Web Text)

Max Activating Token Index: 940

Text #8

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4670. Min Act: -0.1700

Data Index: 6991404 (Open Web Text)

Max Activating Token Index: 539

Text #9

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.3676. Min Act: -0.1700

Data Index: 8501873 (Open Web Text)

Max Activating Token Index: 731

Text #10

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4396. Min Act: -0.1700

Data Index: 5761623 (Open Web Text)

Max Activating Token Index: 390

Text #11

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4333. Min Act: -0.1700

Data Index: 1938638 (Open Web Text)

Max Activating Token Index: 117

Text #12

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4304. Min Act: -0.1700

Data Index: 6737269 (Open Web Text)

Max Activating Token Index: 305

Text #13

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4117. Min Act: -0.1700

Data Index: 227413 (Open Web Text)

Max Activating Token Index: 218

Text #14

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.3779. Min Act: -0.1700

Data Index: 3460660 (Open Web Text)

Max Activating Token Index: 574

Text #15

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4209. Min Act: -0.1700

Data Index: 2831983 (Open Web Text)

Max Activating Token Index: 664

Text #16

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.4163. Min Act: -0.1700

Data Index: 5210539 (Open Web Text)

Max Activating Token Index: 77

Text #17

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.3908. Min Act: -0.1700

Data Index: 2004191 (Open Web Text)

Max Activating Token Index: 550

Text #18

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.3286. Min Act: -0.1700

Data Index: 2756330 (Open Web Text)

Max Activating Token Index: 624

Text #19

Max Range: 5.8241. Min Range: -5.8241

Max Act: 5.3600. Min Act: -0.1700

Data Index: 4151189 (Open Web Text)

Max Activating Token Index: 457