Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2301 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3019. Min Range: -3.3019

Max Act: 3.3019. Min Act: -0.1700

Data Index: 3569410 (Open Web Text)

Max Activating Token Index: 603

Text #1

Max Range: 3.3019. Min Range: -3.3019

Max Act: 3.0085. Min Act: -0.1700

Data Index: 1653640 (Open Web Text)

Max Activating Token Index: 236

Text #2

Max Range: 3.3019. Min Range: -3.3019

Max Act: 3.0118. Min Act: -0.1700

Data Index: 8053249 (Open Web Text)

Max Activating Token Index: 946

Text #3

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8582. Min Act: -0.1700

Data Index: 3197420 (Open Web Text)

Max Activating Token Index: 303

Text #4

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8554. Min Act: -0.1700

Data Index: 4351346 (Open Web Text)

Max Activating Token Index: 615

Text #5

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8674. Min Act: -0.1700

Data Index: 4374526 (Open Web Text)

Max Activating Token Index: 114

Text #6

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8701. Min Act: -0.1700

Data Index: 2598528 (Open Web Text)

Max Activating Token Index: 352

Text #7

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8803. Min Act: -0.1700

Data Index: 8152479 (Open Web Text)

Max Activating Token Index: 819

Text #8

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8579. Min Act: -0.1700

Data Index: 4737797 (Open Web Text)

Max Activating Token Index: 347

Text #9

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8283. Min Act: -0.1700

Data Index: 6764919 (Open Web Text)

Max Activating Token Index: 159

Text #10

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8349. Min Act: -0.1700

Data Index: 6229530 (Open Web Text)

Max Activating Token Index: 618

Text #11

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8059. Min Act: -0.1700

Data Index: 7824435 (Open Web Text)

Max Activating Token Index: 593

Text #12

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8211. Min Act: -0.1700

Data Index: 5193899 (Open Web Text)

Max Activating Token Index: 177

Text #13

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8172. Min Act: -0.1700

Data Index: 5782629 (Open Web Text)

Max Activating Token Index: 274

Text #14

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8391. Min Act: -0.1700

Data Index: 1449393 (Open Web Text)

Max Activating Token Index: 546

Text #15

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8097. Min Act: -0.1700

Data Index: 7642221 (Open Web Text)

Max Activating Token Index: 836

Text #16

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.7996. Min Act: -0.1700

Data Index: 907765 (Open Web Text)

Max Activating Token Index: 384

Text #17

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.7856. Min Act: -0.1700

Data Index: 1635846 (Open Web Text)

Max Activating Token Index: 440

Text #18

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.8205. Min Act: -0.1700

Data Index: 1666998 (Open Web Text)

Max Activating Token Index: 581

Text #19

Max Range: 3.3019. Min Range: -3.3019

Max Act: 2.7961. Min Act: -0.1700

Data Index: 4441757 (Open Web Text)

Max Activating Token Index: 887