Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2658 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0441. Min Range: -3.0441

Max Act: 3.0441. Min Act: -0.1700

Data Index: 4693070 (Open Web Text)

Max Activating Token Index: 975

Text #1

Max Range: 3.0441. Min Range: -3.0441

Max Act: 3.0238. Min Act: -0.1700

Data Index: 5049414 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.0441. Min Range: -3.0441

Max Act: 3.0069. Min Act: -0.1700

Data Index: 2789482 (Open Web Text)

Max Activating Token Index: 22

Text #3

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9818. Min Act: -0.1700

Data Index: 6717514 (Open Web Text)

Max Activating Token Index: 575

Text #4

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9514. Min Act: -0.1700

Data Index: 4339743 (Open Web Text)

Max Activating Token Index: 961

Text #5

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9683. Min Act: -0.1700

Data Index: 2930212 (Open Web Text)

Max Activating Token Index: 363

Text #6

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9678. Min Act: -0.1700

Data Index: 8647519 (Open Web Text)

Max Activating Token Index: 579

Text #7

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9373. Min Act: -0.1700

Data Index: 4286571 (Open Web Text)

Max Activating Token Index: 549

Text #8

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9360. Min Act: -0.1700

Data Index: 6147742 (Open Web Text)

Max Activating Token Index: 668

Text #9

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9369. Min Act: -0.1700

Data Index: 6406704 (Open Web Text)

Max Activating Token Index: 654

Text #10

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9376. Min Act: -0.1700

Data Index: 1982925 (Open Web Text)

Max Activating Token Index: 839

Text #11

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9335. Min Act: -0.1700

Data Index: 1942911 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9375. Min Act: -0.1700

Data Index: 5899865 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9283. Min Act: -0.1700

Data Index: 4576564 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9320. Min Act: -0.1700

Data Index: 4505426 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9166. Min Act: -0.1700

Data Index: 4890870 (Open Web Text)

Max Activating Token Index: 30

Text #16

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9140. Min Act: -0.1700

Data Index: 4165598 (Open Web Text)

Max Activating Token Index: 103

Text #17

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9213. Min Act: -0.1700

Data Index: 6391242 (Open Web Text)

Max Activating Token Index: 60

Text #18

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9135. Min Act: -0.1700

Data Index: 349835 (Open Web Text)

Max Activating Token Index: 83

Text #19

Max Range: 3.0441. Min Range: -3.0441

Max Act: 2.9128. Min Act: -0.1700

Data Index: 6320941 (Open Web Text)

Max Activating Token Index: 969