Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2235 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.7484. Min Act: -0.1700

Data Index: 8559225 (Open Web Text)

Max Activating Token Index: 898

Text #1

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.7225. Min Act: -0.1700

Data Index: 4166390 (Open Web Text)

Max Activating Token Index: 352

Text #2

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.7143. Min Act: -0.1700

Data Index: 3549542 (Open Web Text)

Max Activating Token Index: 519

Text #3

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.7170. Min Act: -0.1700

Data Index: 5227535 (Open Web Text)

Max Activating Token Index: 981

Text #4

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.7198. Min Act: -0.1700

Data Index: 4937147 (Open Web Text)

Max Activating Token Index: 985

Text #5

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6919. Min Act: -0.1700

Data Index: 7114885 (Open Web Text)

Max Activating Token Index: 549

Text #6

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6554. Min Act: -0.1700

Data Index: 3160569 (Open Web Text)

Max Activating Token Index: 474

Text #7

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6309. Min Act: -0.1700

Data Index: 6122132 (Open Web Text)

Max Activating Token Index: 507

Text #8

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6281. Min Act: -0.1700

Data Index: 2343253 (Open Web Text)

Max Activating Token Index: 584

Text #9

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6007. Min Act: -0.1700

Data Index: 7315727 (Open Web Text)

Max Activating Token Index: 500

Text #10

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5801. Min Act: -0.1700

Data Index: 1129306 (Open Web Text)

Max Activating Token Index: 916

Text #11

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6011. Min Act: -0.1700

Data Index: 27582 (Open Web Text)

Max Activating Token Index: 419

Text #12

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.6049. Min Act: -0.1700

Data Index: 3259992 (Open Web Text)

Max Activating Token Index: 496

Text #13

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5764. Min Act: -0.1700

Data Index: 8601446 (Open Web Text)

Max Activating Token Index: 836

Text #14

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5568. Min Act: -0.1700

Data Index: 2454338 (Open Web Text)

Max Activating Token Index: 323

Text #15

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5705. Min Act: -0.1700

Data Index: 1028561 (Open Web Text)

Max Activating Token Index: 613

Text #16

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5604. Min Act: -0.1700

Data Index: 1599089 (Open Web Text)

Max Activating Token Index: 468

Text #17

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5508. Min Act: -0.1700

Data Index: 3275888 (Open Web Text)

Max Activating Token Index: 814

Text #18

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5530. Min Act: -0.1700

Data Index: 1836331 (Open Web Text)

Max Activating Token Index: 89

Text #19

Max Range: 4.7484. Min Range: -4.7484

Max Act: 4.5776. Min Act: -0.1700

Data Index: 256618 (Open Web Text)

Max Activating Token Index: 465