Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 713 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.7438. Min Act: -0.1700

Data Index: 925483 (Open Web Text)

Max Activating Token Index: 319

Text #1

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6860. Min Act: -0.1700

Data Index: 2556886 (Open Web Text)

Max Activating Token Index: 639

Text #2

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.7297. Min Act: -0.1700

Data Index: 6862133 (Open Web Text)

Max Activating Token Index: 214

Text #3

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.7218. Min Act: -0.1700

Data Index: 4624753 (Open Web Text)

Max Activating Token Index: 814

Text #4

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6329. Min Act: -0.1700

Data Index: 6825310 (Open Web Text)

Max Activating Token Index: 51

Text #5

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6714. Min Act: -0.1700

Data Index: 2383236 (Open Web Text)

Max Activating Token Index: 478

Text #6

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6685. Min Act: -0.1700

Data Index: 1748063 (Open Web Text)

Max Activating Token Index: 476

Text #7

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6557. Min Act: -0.1700

Data Index: 543230 (Open Web Text)

Max Activating Token Index: 672

Text #8

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6154. Min Act: -0.1700

Data Index: 4806245 (Open Web Text)

Max Activating Token Index: 970

Text #9

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6318. Min Act: -0.1700

Data Index: 5008358 (Open Web Text)

Max Activating Token Index: 525

Text #10

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6339. Min Act: -0.1700

Data Index: 5355190 (Open Web Text)

Max Activating Token Index: 483

Text #11

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6420. Min Act: -0.1700

Data Index: 2240845 (Open Web Text)

Max Activating Token Index: 61

Text #12

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6623. Min Act: -0.1700

Data Index: 8173655 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6371. Min Act: -0.1700

Data Index: 3603348 (Open Web Text)

Max Activating Token Index: 386

Text #14

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6327. Min Act: -0.1700

Data Index: 5461232 (Open Web Text)

Max Activating Token Index: 288

Text #15

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6394. Min Act: -0.1700

Data Index: 6091275 (Open Web Text)

Max Activating Token Index: 49

Text #16

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6134. Min Act: -0.1700

Data Index: 2805280 (Open Web Text)

Max Activating Token Index: 853

Text #17

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.5798. Min Act: -0.1700

Data Index: 5410978 (Open Web Text)

Max Activating Token Index: 601

Text #18

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.5857. Min Act: -0.1700

Data Index: 2580773 (Open Web Text)

Max Activating Token Index: 602

Text #19

Max Range: 3.7438. Min Range: -3.7438

Max Act: 3.6267. Min Act: -0.1700

Data Index: 6319780 (Open Web Text)

Max Activating Token Index: 484