Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 37 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.7027. Min Act: -0.1700

Data Index: 1173362 (C4 (Web Text))

Max Activating Token Index: 38

Text #1

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.6182. Min Act: -0.1700

Data Index: 109882 (C4 (Web Text))

Max Activating Token Index: 830

Text #2

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.6089. Min Act: -0.1700

Data Index: 308709 (C4 (Web Text))

Max Activating Token Index: 827

Text #3

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.5880. Min Act: -0.1700

Data Index: 253338 (C4 (Web Text))

Max Activating Token Index: 277

Text #4

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.5554. Min Act: -0.1700

Data Index: 632555 (C4 (Web Text))

Max Activating Token Index: 481

Text #5

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.5087. Min Act: -0.1700

Data Index: 863986 (C4 (Web Text))

Max Activating Token Index: 273

Text #6

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4969. Min Act: -0.1700

Data Index: 272937 (C4 (Web Text))

Max Activating Token Index: 230

Text #7

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.5348. Min Act: -0.1700

Data Index: 301559 (C4 (Web Text))

Max Activating Token Index: 397

Text #8

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4882. Min Act: -0.1700

Data Index: 1224350 (C4 (Web Text))

Max Activating Token Index: 53

Text #9

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4683. Min Act: -0.1700

Data Index: 986051 (C4 (Web Text))

Max Activating Token Index: 778

Text #10

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4873. Min Act: -0.1700

Data Index: 752991 (C4 (Web Text))

Max Activating Token Index: 267

Text #11

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4882. Min Act: -0.1700

Data Index: 466088 (C4 (Web Text))

Max Activating Token Index: 698

Text #12

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4553. Min Act: -0.1700

Data Index: 819875 (C4 (Web Text))

Max Activating Token Index: 968

Text #13

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4598. Min Act: -0.1700

Data Index: 341775 (C4 (Web Text))

Max Activating Token Index: 298

Text #14

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4317. Min Act: -0.1700

Data Index: 1125304 (C4 (Web Text))

Max Activating Token Index: 175

Text #15

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4428. Min Act: -0.1700

Data Index: 412758 (C4 (Web Text))

Max Activating Token Index: 238

Text #16

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4245. Min Act: -0.1700

Data Index: 53705 (C4 (Web Text))

Max Activating Token Index: 409

Text #17

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.4143. Min Act: -0.1700

Data Index: 1225608 (C4 (Web Text))

Max Activating Token Index: 601

Text #18

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.3868. Min Act: -0.1700

Data Index: 68799 (C4 (Web Text))

Max Activating Token Index: 697

Text #19

Max Range: 3.7027. Min Range: -3.7027

Max Act: 3.3701. Min Act: -0.1700

Data Index: 1066907 (C4 (Web Text))

Max Activating Token Index: 715