Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1149 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.7369. Min Act: -0.1700

Data Index: 4865697 (Open Web Text)

Max Activating Token Index: 667

Text #1

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.7301. Min Act: -0.1700

Data Index: 6248344 (Open Web Text)

Max Activating Token Index: 668

Text #2

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.6467. Min Act: -0.1700

Data Index: 610102 (Open Web Text)

Max Activating Token Index: 667

Text #3

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.6718. Min Act: -0.1700

Data Index: 5850361 (Open Web Text)

Max Activating Token Index: 875

Text #4

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.6560. Min Act: -0.1700

Data Index: 8024001 (Open Web Text)

Max Activating Token Index: 27

Text #5

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.6227. Min Act: -0.1700

Data Index: 8271001 (Open Web Text)

Max Activating Token Index: 144

Text #6

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5851. Min Act: -0.1700

Data Index: 3619845 (Open Web Text)

Max Activating Token Index: 758

Text #7

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5510. Min Act: -0.1700

Data Index: 4177872 (Open Web Text)

Max Activating Token Index: 690

Text #8

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5681. Min Act: -0.1700

Data Index: 8681530 (Open Web Text)

Max Activating Token Index: 954

Text #9

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5663. Min Act: -0.1700

Data Index: 141437 (Open Web Text)

Max Activating Token Index: 124

Text #10

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5678. Min Act: -0.1700

Data Index: 1665273 (Open Web Text)

Max Activating Token Index: 699

Text #11

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5868. Min Act: -0.1700

Data Index: 6860656 (Open Web Text)

Max Activating Token Index: 140

Text #12

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5731. Min Act: -0.1700

Data Index: 6258134 (Open Web Text)

Max Activating Token Index: 422

Text #13

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5333. Min Act: -0.1700

Data Index: 3697200 (Open Web Text)

Max Activating Token Index: 648

Text #14

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.6005. Min Act: -0.1700

Data Index: 8265557 (Open Web Text)

Max Activating Token Index: 612

Text #15

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5502. Min Act: -0.1700

Data Index: 4265056 (Open Web Text)

Max Activating Token Index: 525

Text #16

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5059. Min Act: -0.1700

Data Index: 8118748 (Open Web Text)

Max Activating Token Index: 776

Text #17

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5169. Min Act: -0.1700

Data Index: 5850363 (Open Web Text)

Max Activating Token Index: 414

Text #18

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5150. Min Act: -0.1700

Data Index: 6419748 (Open Web Text)

Max Activating Token Index: 536

Text #19

Max Range: 4.7369. Min Range: -4.7369

Max Act: 4.5109. Min Act: -0.1700

Data Index: 6741601 (Open Web Text)

Max Activating Token Index: 965