Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1346 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.8147. Min Act: -0.1700

Data Index: 7053800 (Open Web Text)

Max Activating Token Index: 309

Text #1

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.6578. Min Act: -0.1700

Data Index: 7370640 (Open Web Text)

Max Activating Token Index: 496

Text #2

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.6193. Min Act: -0.1700

Data Index: 5317092 (Open Web Text)

Max Activating Token Index: 358

Text #3

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.6380. Min Act: -0.1700

Data Index: 5886338 (Open Web Text)

Max Activating Token Index: 838

Text #4

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5991. Min Act: -0.1700

Data Index: 1793085 (Open Web Text)

Max Activating Token Index: 853

Text #5

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5845. Min Act: -0.1700

Data Index: 1772948 (Open Web Text)

Max Activating Token Index: 379

Text #6

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5901. Min Act: -0.1700

Data Index: 6449227 (Open Web Text)

Max Activating Token Index: 553

Text #7

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5862. Min Act: -0.1700

Data Index: 4201699 (Open Web Text)

Max Activating Token Index: 220

Text #8

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5852. Min Act: -0.1700

Data Index: 1495555 (Open Web Text)

Max Activating Token Index: 645

Text #9

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5673. Min Act: -0.1700

Data Index: 7003587 (Open Web Text)

Max Activating Token Index: 356

Text #10

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5364. Min Act: -0.1700

Data Index: 2169971 (Open Web Text)

Max Activating Token Index: 656

Text #11

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5422. Min Act: -0.1700

Data Index: 6729108 (Open Web Text)

Max Activating Token Index: 591

Text #12

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5365. Min Act: -0.1700

Data Index: 1319971 (Open Web Text)

Max Activating Token Index: 188

Text #13

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5344. Min Act: -0.1700

Data Index: 1252630 (Open Web Text)

Max Activating Token Index: 809

Text #14

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.4939. Min Act: -0.1700

Data Index: 4193254 (Open Web Text)

Max Activating Token Index: 261

Text #15

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.4733. Min Act: -0.1700

Data Index: 5716118 (Open Web Text)

Max Activating Token Index: 843

Text #16

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.4750. Min Act: -0.1700

Data Index: 7217555 (Open Web Text)

Max Activating Token Index: 104

Text #17

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.5058. Min Act: -0.1700

Data Index: 5439435 (Open Web Text)

Max Activating Token Index: 747

Text #18

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.4791. Min Act: -0.1700

Data Index: 6575408 (Open Web Text)

Max Activating Token Index: 195

Text #19

Max Range: 3.8147. Min Range: -3.8147

Max Act: 3.4839. Min Act: -0.1700

Data Index: 3829957 (Open Web Text)

Max Activating Token Index: 934