Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2432 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.3324. Min Act: -0.1700

Data Index: 6267046 (Open Web Text)

Max Activating Token Index: 520

Text #1

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.1558. Min Act: -0.1700

Data Index: 1544 (Open Web Text)

Max Activating Token Index: 761

Text #2

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.1267. Min Act: -0.1700

Data Index: 7654480 (Open Web Text)

Max Activating Token Index: 384

Text #3

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.1707. Min Act: -0.1700

Data Index: 7857270 (Open Web Text)

Max Activating Token Index: 360

Text #4

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.1196. Min Act: -0.1700

Data Index: 5023908 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.1231. Min Act: -0.1700

Data Index: 8392586 (Open Web Text)

Max Activating Token Index: 908

Text #6

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0795. Min Act: -0.1700

Data Index: 4045677 (Open Web Text)

Max Activating Token Index: 958

Text #7

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0807. Min Act: -0.1700

Data Index: 4125925 (Open Web Text)

Max Activating Token Index: 27

Text #8

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0979. Min Act: -0.1700

Data Index: 5123541 (Open Web Text)

Max Activating Token Index: 474

Text #9

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0933. Min Act: -0.1700

Data Index: 7377124 (Open Web Text)

Max Activating Token Index: 62

Text #10

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0828. Min Act: -0.1700

Data Index: 3649350 (Open Web Text)

Max Activating Token Index: 500

Text #11

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0612. Min Act: -0.1700

Data Index: 5606807 (Open Web Text)

Max Activating Token Index: 699

Text #12

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0461. Min Act: -0.1700

Data Index: 1134293 (Open Web Text)

Max Activating Token Index: 594

Text #13

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0633. Min Act: -0.1700

Data Index: 8499100 (Open Web Text)

Max Activating Token Index: 252

Text #14

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0530. Min Act: -0.1700

Data Index: 7892870 (Open Web Text)

Max Activating Token Index: 98

Text #15

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0874. Min Act: -0.1700

Data Index: 7655117 (Open Web Text)

Max Activating Token Index: 781

Text #16

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0575. Min Act: -0.1700

Data Index: 4378490 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0718. Min Act: -0.1700

Data Index: 6131843 (Open Web Text)

Max Activating Token Index: 484

Text #18

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0485. Min Act: -0.1700

Data Index: 2004813 (Open Web Text)

Max Activating Token Index: 431

Text #19

Max Range: 3.3324. Min Range: -3.3324

Max Act: 3.0179. Min Act: -0.1700

Data Index: 6042575 (Open Web Text)

Max Activating Token Index: 228