Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1302 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.1690. Min Act: -0.1700

Data Index: 2308495 (Open Web Text)

Max Activating Token Index: 934

Text #1

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.1782. Min Act: -0.1700

Data Index: 4251303 (Open Web Text)

Max Activating Token Index: 883

Text #2

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.1399. Min Act: -0.1700

Data Index: 3504260 (Open Web Text)

Max Activating Token Index: 401

Text #3

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.1648. Min Act: -0.1700

Data Index: 4778768 (Open Web Text)

Max Activating Token Index: 698

Text #4

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.1343. Min Act: -0.1700

Data Index: 8234240 (Open Web Text)

Max Activating Token Index: 773

Text #5

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0718. Min Act: -0.1700

Data Index: 3294015 (Open Web Text)

Max Activating Token Index: 488

Text #6

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0817. Min Act: -0.1700

Data Index: 470877 (Open Web Text)

Max Activating Token Index: 872

Text #7

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0589. Min Act: -0.1700

Data Index: 6912998 (Open Web Text)

Max Activating Token Index: 744

Text #8

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0591. Min Act: -0.1700

Data Index: 1380770 (Open Web Text)

Max Activating Token Index: 663

Text #9

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0382. Min Act: -0.1700

Data Index: 3502820 (Open Web Text)

Max Activating Token Index: 653

Text #10

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0405. Min Act: -0.1700

Data Index: 470878 (Open Web Text)

Max Activating Token Index: 203

Text #11

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0699. Min Act: -0.1700

Data Index: 470882 (Open Web Text)

Max Activating Token Index: 66

Text #12

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0245. Min Act: -0.1700

Data Index: 7918897 (Open Web Text)

Max Activating Token Index: 91

Text #13

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0484. Min Act: -0.1700

Data Index: 470879 (Open Web Text)

Max Activating Token Index: 237

Text #14

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0077. Min Act: -0.1700

Data Index: 1912173 (Open Web Text)

Max Activating Token Index: 384

Text #15

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0098. Min Act: -0.1700

Data Index: 4374962 (Open Web Text)

Max Activating Token Index: 910

Text #16

Max Range: 4.1782. Min Range: -4.1782

Max Act: 4.0502. Min Act: -0.1700

Data Index: 6980322 (Open Web Text)

Max Activating Token Index: 421

Text #17

Max Range: 4.1782. Min Range: -4.1782

Max Act: 3.9912. Min Act: -0.1700

Data Index: 4922811 (Open Web Text)

Max Activating Token Index: 878

Text #18

Max Range: 4.1782. Min Range: -4.1782

Max Act: 3.9950. Min Act: -0.1700

Data Index: 845458 (Open Web Text)

Max Activating Token Index: 852

Text #19

Max Range: 4.1782. Min Range: -4.1782

Max Act: 3.9808. Min Act: -0.1700

Data Index: 2136491 (Open Web Text)

Max Activating Token Index: 970