Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2339 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0135. Min Range: -4.0135

Max Act: 4.0135. Min Act: -0.1700

Data Index: 8263177 (Open Web Text)

Max Activating Token Index: 21

Text #1

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.9715. Min Act: -0.1700

Data Index: 319534 (Open Web Text)

Max Activating Token Index: 838

Text #2

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.9726. Min Act: -0.1700

Data Index: 3004324 (Open Web Text)

Max Activating Token Index: 1002

Text #3

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.9520. Min Act: -0.1700

Data Index: 2565488 (Open Web Text)

Max Activating Token Index: 787

Text #4

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.9446. Min Act: -0.1700

Data Index: 1556702 (Open Web Text)

Max Activating Token Index: 436

Text #5

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8957. Min Act: -0.1700

Data Index: 4682632 (Open Web Text)

Max Activating Token Index: 791

Text #6

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.9390. Min Act: -0.1700

Data Index: 6421558 (Open Web Text)

Max Activating Token Index: 904

Text #7

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8407. Min Act: -0.1700

Data Index: 8224139 (Open Web Text)

Max Activating Token Index: 164

Text #8

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8559. Min Act: -0.1700

Data Index: 4225601 (Open Web Text)

Max Activating Token Index: 171

Text #9

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8434. Min Act: -0.1700

Data Index: 1534780 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8502. Min Act: -0.1700

Data Index: 7557988 (Open Web Text)

Max Activating Token Index: 201

Text #11

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8343. Min Act: -0.1700

Data Index: 1319457 (Open Web Text)

Max Activating Token Index: 50

Text #12

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8171. Min Act: -0.1700

Data Index: 3894215 (Open Web Text)

Max Activating Token Index: 27

Text #13

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8169. Min Act: -0.1700

Data Index: 3314329 (Open Web Text)

Max Activating Token Index: 376

Text #14

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8074. Min Act: -0.1700

Data Index: 8103199 (Open Web Text)

Max Activating Token Index: 402

Text #15

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.7918. Min Act: -0.1700

Data Index: 6903638 (Open Web Text)

Max Activating Token Index: 833

Text #16

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.8040. Min Act: -0.1700

Data Index: 7684032 (Open Web Text)

Max Activating Token Index: 721

Text #17

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.7908. Min Act: -0.1700

Data Index: 7991513 (Open Web Text)

Max Activating Token Index: 704

Text #18

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.7783. Min Act: -0.1700

Data Index: 1184393 (Open Web Text)

Max Activating Token Index: 493

Text #19

Max Range: 4.0135. Min Range: -4.0135

Max Act: 3.7837. Min Act: -0.1700

Data Index: 530733 (Open Web Text)

Max Activating Token Index: 386